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A Coding Theorem for Multiple Access 
Channels With Correlated Sources 


By DAVID SLEPIAN and JACK KEIL WOLFt 
(Manuscript received February 22, 1973) 


A communication system ts studied in which two users communicate 
with one receiver over a common discrete memoryless channel. The infor- 
mation to be transmitted by the users may be correlated. Their information 
rates are described by a point in a suitably defined three-dimensional 
rate space. 

A point in this rate space ts called admissible if there exist coders and 
decoders for the channel that permit the users to transmit information over 
it at the corresponding rates with arbitrarily small error probability. The 
closure of the set of all admissible rate points is called the capacity region, 
C, and is the natural generalization of channel capacity to this situation. 

In this paper we show that @, which depends only on the channel, ts 
convex and we give formulas to determine it exactly. Several simple 
channels are treated in detail and their capacity regions given explicitly. 


I. INTRODUCTION 


The mathematical theory of communication has been concerned, for 
the most part, with the reliable transmission of information from a 
single information source to a single user. An extensive literature exists 

t J. K. Wolf is Professor of Electrical Engineering at the University of Massachu- 


setts, Amherst, Mass. Partial support for his research on this paper was furnished 
by the Air Force Office of Scientific Research under contract F-44620-72-C-0085. 
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on this problem: the basic concepts are contained in the classic papers 
of Shannon.! 

In this work we consider the case in which messages from a set of 
information sources are communicated over a common channel to a 
single receiver. We impose constraints on the encoding techniques 
which can be employed. 

A precise formulation of the problem is presented in a subsequent 
section. Here we describe in less mathematical terms the type of 
problem considered. 

A particular multiple access communication channel with two inputs 
and one output is shown in Fig. 1. Here the two inputs, X; and Xe, and 
the output Y each take values from the set {0, 1}. The conditional 
probability of the output Y for each of the four possible input pairs 
(Xi, X2) is also shown in the schema at the right in Fig. 1. 

It is clear that if the transmitters can cooperate with each other 
they can transmit without error one bit per channel use by transmitting 
either the pair (X1 = 0, Xz = 0) or the pair (X,; = 1, X2 = 1). Such 
would be the case if a common binary source were connected to both 
inputs without any coding. If a message is to be transmitted by con- 
necting it to only one input, say to input 1, and if the other input is 
unaware of the message, then even if no information is to be transmitted 
through input 2, the information rate for input 1 must be substan- 
tially less than one bit per channel use in order to achieve reliable 
communication. If two independent messages are to be connected 
separately to the inputs—message 1 to input 1, message 2 to input 2— 
the situation is even more difficult. 

A general configuration that we consider is shown in Fig. 2. Three 
sources emitting statistically independent messages at rates Ro, Ri, and 
R, are connected to a multiple access channel via two encoders. The 
messages from source 1 and source 0 are inputs to encoder 1 and its 
output is connected to one of the input terminals of the channel. 


0 0 
X, € {o.1} 
MULTIPLE ACCESS : 


COMMUNICATION 
CHANNEL 








INPUT 1 


INPUT 2 





Fig. i—A multiple access channel. 
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CHANNEL 





SOURCE 2 ENCODER 2 


Fig. 2—Multiple access channel with correlated sources. 


Encoder 2 has as inputs the messages from both source 2 and source 0 
and its output is connected to the other input terminal of the channel. 
The channel output is connected to a decoder which estimates the three 
source messages. It is convenient to represent the rates of the three 
message sources by a point in a three-dimensional rate space. 

For each given channel of the sort just described, there are certain 
rate triplets, Ro, Ri, R2, for which it is possible to attain arbitrarily 
small probability of error in the system output by using sufficiently 
clever encoding and decoding schemes. For other points in the rate space 
this is not possible. We call the closure of the set of rate points for 
which the error probability can be made arbitrarily small the admissible 
rate region or the capacity region for this channel. It is a natural general- 
ization to the multiple access channel of the channel capacity that is 
associated with the more commonly studied channel having a single 
input and a single output. 

The main result of this paper is a complete determination of the 
capacity region @. A typical case is shown in Fig. 3. The region always 
lies in the first octant and is bounded by the planes Ry = 0, R: = 0, 





Fig. 3—An admissible rate region. 
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R, = 0 and a convex surface. The equations that describe € will be 
shown to involve various conditional and unconditional mutual infor- 
mations. This is analogous to the single-user channel where the capacity 
is calculated from a mutual information. 

Problems resembling ours have been treated by several authors. 
Shannon,? and then Van der Meulen,’ consider a two-way channel with 
two inputs and two outputs. The configuration of the encoders and 
decoders is different than in our model, so that the problems are not the 
same. One similarity, however, is that the two sources are described by 
a pair of rates which are represented by a point in rate space. For 
certain points, encoders and decoders exist for which the probability of 
error can be made as small as desired. 

The multiple access channel has been investigated by Liao,‘ Van der 
Meulen,® and Ahlswede.® Liao‘ and Ahlswede® both prove a coding 
theorem and a converse for the case of independent sources. Our 
results reduce to theirs for the case Ry = 0. Correlation in the sources 
adds a totally new dimension to the problem (and literally to the region 
of admissible rates). 

A problem which is the dual of the one considered here is the broad- 
cast channel investigated by Cover’ and Bergmans.® There, a channel 
with one input and two outputs is considered along with a single 
encoder and two decoders. Again the concept of an admissible rate 
region applies. 

A brief outline of the paper follows. In Section II a detailed problem 
formulation is presented. Section III summarizes the main results of 
the paper and gives some examples. Sections IV and V and the as- 
sociated appendixes give details of the derivation of a coding theorem 
and a converse. A more useful description of the admissible rate region 
is given in Section VI. We conclude in Section VII with some generaliza- 
tions and comments. 


II, PROBLEM FORMULATION 


Consider the block diagram shown in Fig. 4. The three sources are 
described by a three-dimensional rate vector R = (Ro, Ri, Re) with non- 
negative components. For a fixed positive integer N, we define the 
components of the vector M by 


M = M(R, N) = (Mo, Mi, M2), (la) 
M,=Te®"1, 7 =0,1,2, (1b) 


where Tz! is the smallest integer greater than or equal to x. Every NV 
time units the sources produce a triplet of numbers (2, j, k) that are the 
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U; =i*e fi.2, ke 


% Up = ite fia wo Mg 
MULTIPLE Y =ye(Y) 
ACCESS DECODER 


CHANNEL U5 =k*e fia wa, Mg 


Fig. 4—Notation for multiple access channel. 


corresponding values of the random variables (Uo, U1, U2). These 
random variables are assumed to be statistically independent and 
uniformly distributed over the rectangular lattice of dimensions 
My X Mi X M3. That is, their joint probability distribution is 
Pony, (t, j,k) = Pr[Uo = 7, U1 = j, U2 = kJ = 1/MoM iM, 
te d;2; ane , Mo) = I), 
Te (1, 2, << , M1) eh, 
kE (1, 2, is , M2) = T1.. 


The channel is a probabilistic mapping which every unit of time maps 
a pair of real numbers (21, %2) to the real number y. The real numbers 
2X1, L2, and y belong to the finite alphabets 1, X2, and YY, respectively. 
The mapping is governed by the conditional probability distribution 
Pyix,x,(y|21, 22) for all x1 in 1, x2 in Xe, and y in Y. Here we describe 
the inputs by the pair of random variables (X1, X2) and the output by 
the random variable Y. Throughout this paper, it will be assumed that 
Pyix,x, 1s specified a priort and cannot be altered. 

To describe how the channel processes sequences of N input pairs, 
we define the N-vectors 


(2) 


X1 = (11, 2, +++ , Lin), xpG (Xi)”, 
XxX. = (aa, U2, °°" y Xen); X2 S (No), (3) 
tiie (y1, Yo,°"" yw), y cm (Y)%, 
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and in a similar way the corresponding random vectors Xi, X2, and Y. 
Here (1) is the set of all N-vectors whose components are in X,. The 
sets (X2)* and (Y)* are defined analogously. We assume the channel 
is stationary and memoryless; that is, 


N 
PYYk.x,(¥ | X1, X2) = It Pyix,x,(yt| Cit, 24). (4) 


The superscript N on the joint probability distribution indicates the 
dimension of the vectors. 

The encoders are deterministic mappings from the source outputs to 
channel input vectors. Encoder 1 is a mapping from the source pair 
(zt, 7) to an N-vector x1 € (%1)*. The functional form for this mapping 
is written 


xX1=fy(i,7), tE€lo, gEh, x1 € (%1)%. (5) 


Similarly, encoder 2 is a mapping from the source pair (7, k) to the 
N-vector X2 € (%X2)*. The functional form for this mapping is written 


X= gn(i,k), t€ Io, KE Ie, x2 € (X2)%. (6) 


The collection of (My X M1 + M»o X M2) N-vectors which result from 
these mappings is called a code of block length N. Usually, we will adopt 
the more suggestive notation X1;; and X2,, instead of fy(z, 7) and gy(i, k). 

To summarize the operation of the sources, encoders, and channel we 
note that: 


(t) Every N time units, the three sources produce a triplet (2, 7, k). 
(17) The two encoders act upon the source outputs to produce the 
two N-vectors X1:; and Xoiz. 

(zz) The components of these vectors are impressed upon the 
channel, one pair of inputs each time unit. Corresponding to 
each pair of inputs the channel produces an output, so that in 
the N time units the channel produces an output N-vector, y. 


The decoder is a deterministic mapping from the vector y to the 
triplet (2*, j*, k*) where 7* € Ip, 7* € Th, k* © Iz. We describe this 
mapping by (i*, j*, k*) = hy(y). The triplet of decoder outputs is 
denoted by the vector random variable (Uo, Ui, U2). 

The deterministic mappings (fy, gy, hy) will be called a coding. A 
coding with rate vector R = (Ro, Ri, Rz) and block length N will be 
denoted by Cy(R). For a given coding, we can in principle calculate 
the probability of the error event &, where &° (the complement of &) 
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is defined as 
&° = event {Uj = Uo and U;= Ui, and U3;= U2}. (7) 


For the coding C'y(R), we denote the probability of the error event 
(hereafter called the probability of error) by P.(Cy(R)). 

A rate vector R will be said to be admissible if, for every « > 0, there 
exists a positive integer N and a coding C'y(R) such that P.(Cy(R)) Se. 
The closure of the set of admissible rate vectors is called the admissible 
region or capacity region, and is denoted by @. Our purpose is to specify 
€ for an arbitrary, discrete, memoryless, multiple access channel. 


III. SUMMARY OF RESULTS AND EXAMPLES 
The main results of this paper are two alternative descriptions of the 
admissible rate region @ for any discrete memoryless channel. The 
proofs that these yield the correct region are contained in the remaining 
sections of the paper. Here we discuss only the simplest of the results. 
We shall have much need of conditional mutual information expres- 
sions in the sequel. We remind the reader of the definition 


WAsBIC) = EEE Paw Woe pi Pacg ey © 


Here 
Pasc(i, j,k) = Pr[Aa = ta, Bs = jp, Cy = ky, 
a=1,2,---,L;8=1,2,---,Ms7y=1,2,---,N] 

is the joint distribution function of the discrete random variables 
A, Ao, +++, Ax, Bi, Bo, +++, Bu, Ci, Co, «++, Cy. The conditional 
distributions Pasic(i, j|k), etc., are defined in the usual way. 

Let us return now to consider a discrete memoryless channel with 
input alphabets X1 and X2, output alphabet Y, and transition prob- 


abilities Pyix,x,(y|21, V2), 1 © V1, Xe © Xe, y € Y. Let Z bea random 
variable which takes on values in the set 


= {1,2,---, M}. (9) 


From any set of three distributions Px,\z(x1|z), Px,)z(v2|z), and P2(2), 
t1 © Xi, x2 © Le, 2 € 3, form the joint distribution 


Pzx,xey (2, £1, 2, Y) 
= Pz(z)Pxy\2(a1|2)Px.j2(x2|2)Pyix,x.(y|21, £2). (10) 


Now denote by ®(Pzx,x.v) the set of vectors R = (Ro, R:, Re) such that 
0S Ri S W(X; Y|X2, Z), (11a) 
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0S R, S 1(X2; Y|X,, 4), (11b) 
0 < R, + R, Ss I(X, X33 Y|Z), (11c) 
0 < Ro + Ry + R, < T(X, Xo; Y), (11d) 


where the mutual informations are computed according to (8) using the 
joint distribution (10). This region is a polyhedron such as is shown in 
Fig. 7, Appendix I. Then the admissible rate region @ is given by 


© = closure of the convex hull U ®(Pzx,x.y), (12) 


where the union is taken over all possible choices of Px,iz, Px.1z, and 
Pz, and all values of M, the size of the 3 alphabet.t 

To obtain the intersection of the admissible rate region © with the 
plane Ry = 0, the size of the alphabet 3 can be set equal to 1. The 
random variable Z no longer appears in the equations. For Ry = 0, we 
then define ®(Px,, Px,) as the set of vector R = (0, Ri, R:) such that 


0s Rf: S$ 1(X1; Y|X2), (13a) 
0S Rk. S 1(X2; Y|X)), (13b) 
OS Rit Re S 1(X1, Xo; Y). (13¢) 


Then 
C|z-0 = closure of the convex hull of U ®(Px,, Px,), (14) 


where the union is taken over all possible choices for the unconditional 
distributions Px, and Px,. This is the solution found by Liao‘ for 
uncorrelated sources. 

Other equations for specifying the region © are given in Section VI. 
They involve the calculation of mutual informations among long 
sequences of random variables and thus do not appear to be useful for 
computation. 

Quite generally, © is convex. It is always bounded by portions of the 
three coordinate planes and a surface which encloses a finite volume 
in the first quadrant of rate space. If R = (Ro, Ri, Re) is in ©, then for 
any 5 = (60, 61, 62) satisfying 0 S 6; S R:, 7 = 0, 1, 2, the rate vector 
5 is also in @. 

In the remainder of this section, some simple examples are presented 
for which the admissible rate region has an explicit characterization. 


t We suspect that it suffices to consider only values of M < "e®1, but have not 
been able to prove this conjecture. 
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Example 1 (Multiplier Channel) 


Both the inputs, X, and X», and the output, Y, for this channel 
take values 0 and 1. The output is the product of the two inputs. For- 
mally, XX = Xe = Y => {0, 1} and Py,x,x,(0|0, 0) > Pyix,x,(0/0, 1) 
= Pyix,x,(0/1, 0) = Pyix,x,(1/1,1) = 1, and all other conditional 
probabilities are zero. Note that the channel is deterministic. 

The pyramid described by the planes 

Ro = 0, R, = 0, R, = 0, 
R,+Ri+ Ro = log 2 
must contain the admissible rate region @, as is seen from (11d) since 
0S Ro + Rit Ro S 1(X1, X2; Y) S log 2. But the rate vectors 
R; = (log 2, 0, 0), Re = (0, log 2, 0), and R; = (0, 0, log 2) are all 
admissible by the following strategies: 
Ri: Choose N = 1, Mo = 2, M1 = Mz. = 1 and use code words 
Yiu = O, Tir = 1, Yau = O, Fey = 1. 

R.: Choose N = 1, My = 1, Mi = 2, M2 = 1 and use code words 
tin = 0, Mw = 1, Xn = 1. 

R;: Choose N = 1, Myo = 1, M1 = 1, M2 = 2 and use code words 
Ti = 1, Ya = 0, tov = 1. 


(15) 


The probability of error for these codes is zero. Since the convex hull 
of these three rate points and the origin is the set bounded by the 
planes (15), the capacity region @ is as shown in Fig. 5. 

By similar arguments, we find that Fig. 5 gives the region of admis- 
sible rates for many other binary-input, binary-output deterministic 





Ro 


Fig. 5—Admissible rate region for the multiplier channel. 
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TABLE I—Py}x,x,(y|2, %2) 


y 
L1X2 0 1 2 3 

0 0 1—p p/3 p/3 p/3 

0 1 p/ LD p/3 p/3 

1 0 p/3 p/3 Lp p/3 

1 1 p/3 p/3 p/ 1— 


channels (ones with all transition probabilities equal to zero or one). 
Degenerate cases exist, however, in which the region © reduces to a 
portion of a plane. For example, if Pyix,x,(0|0, 0) = Pyix,x,(0|0, 1) 
= Pyix,x,(1|1, 0) = Pyix,x,(1|1, 1) = 1 and all other probabilities 
are zero, it is easy to verify that C = {R = (Ro, Ri, R2):0S Ro + Ri 
S log 2, Re = 0, Ri, Ro = O}. 


Example 2 (Symmetric Noisy Channel) 


Let Xi = Xo = {0, ike Y = {0, 1, 2, 3} and let Pyix,x,(y| 1, X2) be 
given as shown in Table I. Let M be as in (9). Define Pz(z;) =yi:, 


Px,1z(0|z:) = ai, Px.iz(0|z:) = Bi, 7 = 1, 2, --- , M. Straightforward 
calculations then yield 
M 
I(X1; Y|X2, Z) = my vil fila: p) — K(p)), (16) 
M 
I(X2; Y|X1, Z) = a vil fi(Bi, p) — K(p)), (17) 
M 
I(X1, X33 Y |Z) = my vil folai, B:,p) ist K(p)), (18) 
where 


(8, p) = Sp log 2 2+ (a-+Za-9) 
1 ; 
X log ———————— + (56+ - p)(l - 5) 
(1 p)b +2 — 8) (3 ) 
Sg ee 1) 
go+ (1 — pl — 8) 


K(p) = (1 — p) log er - + plog = (20) 
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and 
1 
fo(a, B, p) = ja — p)ap + 5 (dl — 28) log 


(1 — p)ab + § (1 — a8) 


+ (d- pia — 2) +20 -a +6) 
1 


xX log — 
(1 — p)a(1 — 8) + § 1 — a + af) 


+ (pe — a) +30 - 6 + a8)) 
1 


ee 
(1 — p)B(L — a) + 8 — 6 + a8) 


+(a-pa—o0- 9) +2 +626) 
X ee a ee (21) 
(1 — p)(1 ~ a)(1 — 6) + 5 (@ + 6 — a8) 


It is easy to show that fi(6, Pp) = fi, P); frla, B, Pp) Ss fala, 3) Pp) 
= log 4. Therefore, the three mutual informations in (16), (17), and 
(18) are simultaneously maximized by setting a: = 6; = 3, 7 = 1, 
2,-°-+, M. Furthermore, 


I(X1, Xo; Y) = H(Y) — H(Y|X1iX2) = A(Y) — K(p) S log 4 — K(p) 


with equality when a; = 8; = 3, 7=1, 2, ---, M. Thus all four 
mutual informations, [(X1; Y|X2, Z), I(X2; Y|X1, Z), [(Xi, X2; Y|Z), 
and [(X1, X2; Y), are maximized for the same choice of the parameters 
ai, Bi, and y;, and the maximum values are independent of M. The 
capacity region for this channel then is given by 


O05 fis, p) — K(p), (22) 
05 Rk: S fig, p) — K(p), (23) 
OS Ri t+ Ai + Re S log4 — K(p). (24) 


This region is shown in Fig. 6. 


IV. EXISTENCE OF CODINGS WITH SMALL P, 


In this section we outline a proof of the existence of codings which 
have vanishingly small probability of error for certain values of the 
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f, (1/2, p) — K (p) —~ 


Ry 


~ Ff, (1/2, p) = K (p) 


Fig. 6—Admissible rate region for the symmetric noisy binary channel. 


rate vector R and sufficiently large block length N. Tedious details are 
relegated to the appendixes. A random coding argument is used. We 
calculate the average probability of error for an ensemble of codings, 
then argue that there must exist at least one member of the ensemble 
having error probability as small as this average. Actually, we can only 
compute an upper bound for this average error probability, but this 
bound is sufficiently small for our purposes. 

For every coding, we shall use the same form of decoder mapping. 
Assume for the moment that the block length N, the rate vector R, and 
the encoder functions fy(z, 7) = X1i; and gy(2, k) = Xeu, are fixed. For 
each y € (Y)*%, the decoder computes the Mo X Mi X M2 numbers 


PYYkx.(Y |X1ij, Xow), = EC To, IE, KE Ip. 


Then h(y) = (¢., jo, ko) if and only if (¢,, jo, ko) is the smallest triplet 
(in lexicographic order) such that 


PYHX,xAY |Xrigion X2igko) 2 PY kx. (y |X1ij, Xx) (25) 


for all (¢, 7, k). Such a decoder mapping achieves a maximum likelihood 
decision among the possible source outputs. 

We now describe the class of codings for which we obtain an upper 
bound to the average probability of error. It is specified by two positive 
integers, K and N = KL, where Lis a positive integer, by a rate vector 


MULTIPLE ACCESS CHANNELS 1049 


R, and by a particular probability distribution for the random variables 
X, X2, and Z. The vectors X; and X; are K-dimensional and take on 
values in (%1)* and (X2)* respectively, the spaces of channel input 
K-vectors. The random variable Z takes values from an alphabet 3 of 
size M as in (9). The joint distribution of these quantities is restricted 
to have the form 


Px,(2, X1, X2) = PxPz(K1|2) P¥la(x2|z)Pz(z) (26) 


and we denote this collection of distributions by @x. A class of codings 
is thus specified by K, N = KL, R, and a P&&x, € @x. 

Now let K, N = KL,R, and Px, € @x be given. A set of N-dimen- 
sional code vectors X111, ++ , X1131,, X211) °** , X21, 1n the corresponding 
ensemble of codings is obtained as follows. Choose a sample, say 2, 
from the distribution Pz(-). Next independently choose M, K-vectors 
from P¥)2(-|z) and then M, K-vectors from Px)z(-|z). These are 
respectively the first K components of the N-vectors X11, +°* , X13); 
Xo11, °° * » X21Mo- 

To obtain the next K components of the code words, independently 
choose a new sample z from Pz(-) and repeat the process. After a total 
of L drawings from Pz(-) the specification of the N-vectors X11, --: , 
X11m,, Xo11, ‘++ , X21m, is complete. The entire process is then repeated 
to obtain the remaining code words—those with second subscript 
equal to 2, 3, --- , Mo. 

We now seek an upper bound to the average probability of error for 
the codings in this ensemble, an average in which the probability of 
error for each particular coding is weighted in accordance with its 
probability of occurrence in the ensemble. We denote this average prob- 
ability of error by P.(N, P$x,) and we denote by Peii,j,4(N, P#x,) 
the average probability of error given that the source triplet (i, 7, k) 
was presented for transmission. A useful result is 
Theorem 1: The average probability of error conditioned on the source 
triplet (2, 7, k) has an upper bound 


4 
Paiz nN, Px.) Ss PS exp {—N[E (pa, Px.) _ pelea li, (27) 
a=1 


where 0 S pa S 1, a = 1, 2, 3, 4, 


Ri, a= 1 
5 _ |he, a=2 
Ka = Rit Ra, a=3 ce 


Ro a Rit Ra, 


R 
I 
tc 
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and 


Ei(ox Px.) = — 3g aD LL PeHelus|2)Pel@) 
 (Z PRPC |2) (PHkaxs(y |xsx)) Or) I+, (20a) 


Balps, Pia.) = — % EL LY PEHelasle)Pal@) 
x (X Pile(a| 2) (PYa(y |x, 2)) +09) 1405, (29b) 


Ba(os, Pix.) = — ze bn XL Pele) 
x (2 X P¥H2(K1| 2) Po (x2 |Z) 
X (Pifa(y|xams))/+rO) 92, (296) 


Bu(os, PAR) = — ze bE (LE PH, ms) 


Xx (P¥k.x,(y | x1, Xp))1/C+a)) 1+ p4— 


e-N (R1i+R2) 4 


(29d) 


A proof of this theorem is given in Appendix A. It follows closely the 
proof given in Gallager® for the single-input, single-output channel. 

Since the bound proved in the theorem is independent of the triplet 
(zt, 7, k), we see that this same bound applies to the unconditioned 
average probability of error P.(N, P&Qx,). Finally, for fixed N = KL, 
and P§x,, there must be at least one coding in the ensemble with 
probability of error no greater than the average probability of error. 
Thus we have 


Theorem 2: For every positive integer K, for every positive integer N that is 
an integral multiple of K, for every joint distribution Px, of form(26), 
and for every rate vector R, there exists a coding Cn(R) such that 


lA 


4 
P.(Cx(R)) S$ _¥ exp {—NCEa(pa, Px) — peRe]} (80) 
for all pa, 0 S pa S$ 1, a = 1, 2, 3, 4. The E, and R, are given by (28) 
and (29). 

For a given Px, and for certain values of the rate vector R, the 
upper bound decreases exponentially in N. For these values of R, by 
making N sufficiently large, we can insure a small probability of error. 
We now determine for what rate vectors this is the case. 


MULTIPLE ACCESS CHANNELS 1051 


Define ®(P§Xx.y) as the set of rate vectors R for which 


0<R< eT Kas YX, Z) (31a) 
0S Ri < 4 1(K ¥[%,, Z) (31b) 
0<R +R < UKs, X,; ¥|Z) (31c) 
0S Rot Rit Re < ZU(Ks, Xs ¥), (31d) 


where the mutual informations are evaluated under the distribution 
Px yz, Xi, Xo, y) = PRx.,(z, X1, X2) P¥xk.x,(y |X1, X2), (32) 


where P#x, is given by (26) and P¥k,x, by (4). 
In Appendix B we prove 


Theorem 8: For every « > 0 and every rate vector R C R(P&x,y), there 
exists an Lo and a sequence of codings, Cy(R), such that 


P.(Cy(R)) Se for every N= KL,L2 Lp. (33) 


This theorem holds for all Px, of the form given in (26), that is, 
for all P&x, € Ox. Now define 


x= U_ &(P&x,y), (34) 
P Fx, E0K 
and finally define 
QR =U Gx, (35) 
K 
where K = 1, 2, ---. We then have the following main result: 


Theorem 4: For every « > 0 and for every rate vector R C @, there ewist 
values of K and Lo and a sequence of codings Cy(R) such that 


P.(By(R)) Se for every N= KL,L2 Lo. (36) 


Note that if we use the statement of Theorem 1 instead of Theorem 2, 
Theorem 4 becomes 


Corollary 1: For every « > 0, for every message triplet (2, j, k), and for 
every rate vector R C &, there exist values of K and Ly and a sequence of 
codings Cy(R) such that 


Pai(Cn(R)) Se — for every N= KL, L2 Ly. (37) 
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V. CONVERSE THEOREM 


In this section, we present a series of lemmas and theorems which 
yield a converse to the Coding Theorem 4. Let & denote the closure of 
the region ® given by (35) and let &*° be the complement of & with 
- respect to the first octant. We shall ultimately show that every coding 
Cx(R) with R € ®&* transmits with a probability of error not less than 
a constant 6 > 0 which is independent of K. 

Our notation is as before except that K, instead of N, will be used 
for the block length of a code. Let K, R, and the channel be given. The 
associated vector M with components 


M,=Tek®e1, «a =0,1,2, (38) 


is then determined. We shall no longer be concerned with ensembles of 
codes, but rather fix our attention on some given encoding functions 
Xiij = £(2, 7), Xow = g(t, kb), where 7 © Io, 7 € Kh, and k € Iz. These 
vectors need not be distinct. Then with the source statistics given by 
(2), the given encoding defines a joint probability distribution 


PRnuxxey (1, 4, k, X1, Xa, y) = P¥ikx.(y | Xs, X2) 
x OX) vous (X1 | 2, JQP vou. (Xe | 4, k) Puy, (2, ie k) (39) 


for the random variables in question. Here 


O8? vo. (X1| 2, j) = Ox1x1 43) (40a) 
OM) vou2(Xe | 2, k) = Oxoxoiny (40b) 


where the right-hand terms are Kronecker deltas. Entropies and 
mutual informations can then be calculated from (89) by the usual 
formulas. 

Several more definitions are needed. We shall make use of the rate 
number vector R’ = (Ro, Ri, Rz) given by 


Ri = 4 log M, = Re a =0,1,2 (41) 
and the elementary entropy function 
h(a) =— xlogx — (1 — x) log (1 — 2). (42) 
Finally, we define 
Pa(Cx(R)) = Pr(Ui # Ui] (43) 
P.o(Cx(R)) = Pr[U: # U2] (44) 


P.3(Cx(R)) = Pr CU; # Ui or Us F Ue]. (45) 
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Then, for the probability of error using the coding Cx(R) we have 


P.(Cx(R)) = Pr[Uo # Uo or Uj # Ui or UZ H U2] 
= max [Pa(Cx), Pe(Cx), Pes(Cx)]. (46) 


We now proceed to the first of the lemmas which is a generalization 
of Fano’s inequality (Ref. 9, Theorem 4.3.1). The proof is given in 
Appendix C. 


Lemma 1: For every K and R and for every Cx(R): 
H(U,|Y, Uo, Uz) S Pa(Cx) log Mi + h(Pa(Cx)); (47a) 
H(U2|Y, Uo, U1) S Peo(Cx) log Mz + h(P.2(Cx)); (47b) 
H(U,4, U2|¥, Uo) S Pes(Cx) log (MiMe2) + h(P.3(Cx)); (47) 
H(Uo, U1, U2|¥) S P.(Cx) log (MoMiM2) + h(P.(Cx)). (47d) 


The next lemma, proved in Appendix D, is a generalization of the 
data processing theorem (Ref. 9, Theorem 4.3.3). 


Lemma 2: For every K and R and every coding Cx(R): 


(a), I(U1;¥|Ue2, Uo) S I(Ki; ¥| Xe, Vo); (48a) 
(b), (U2; ¥|U1, Uo) S T(K23 ¥| Xi, Vo); (48b) 
(c), I(U1, U2; ¥| Uo) S (Ki, Xa; ¥| Wo); (48c) 
(d), I(Uo, Us, U2 ¥) S U(%1, XY). (48d) 


Lemma 8: For every K and R and every coding Cx(R) with rate-number 
vector R’: 


(a), KR, — I(%1; ¥|X2, Uo) S Pa(Cx)KR; + h(Pa(Cx)); (49a) 
(b), KR, — I(Xo; ¥|X1, Uo) S Peo(Cx)KRz + h(P.a(Cx)); (49b) 
(c), K(Ri + Re) — I(X1, Xe; ¥| Uo) 
< P.3(Cx)K(Ri + Rz) + h(P.s(Cx)); (49e) 
(d), K(Ro + Ri + Re) — I(K1, Xe; Y) 
< P.(Cx)K(Ro + Ri + Re)+ h(P(Cx)). (49d) 


The proof is given in Appendix E. 

With these lemmas in hand, we return to the matter of establishing 
a converse to Theorem 4. For a given K, R, and encoding C’x(R), there 
is established a joint probability distribution between the random 
variables Y, Xi, Xe, and Uo given by 


Que.x.y (2, X1, Xe, y) = P¥ik.x.(Y | X1X2) Qo (K1 | NOK) ur(Xe | 1)Qu,(2), (50) 
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where P¥ix,x, is given by (4), 


: 4 
Qu, (t) = M,’ (51a) 
- . 1 Mi 
Ql uo(X1 | 2) = mM, a Oxixiiss (51b) 
J= 
; 1 % 
OXY vo (Xe | 2) = i Oa Oxoxorn) (51c) 


tElo, X1 € (X1)*, Xe © (Xe)*. 


Here C'x(R) is defined by the code words xii; and Xen, 7 € Lo 9 € hi, 
k € Ip. We denote by 9x the set of all distributions QY%,x, derived from 
code books, that is, all distributions of the form obtained by summing 
(50) over all y € (Y)¥. 

With K, R, and an encoding C'x(R) now fixed, we define $°(Q{&,x.y) 
to be the set of all vectors S = (So, S:, Se) with non-negative com- 
ponents such that at least one of the following inequalities is satisfied, 


S:> 2 1(Kss Y|U», Xs), (52a) 
S.> UK: Y| Uo, Xs),  (52b) 
Si + 8S: > 21K, X.; ¥|U>), (52c) 
So+ Sit S:> ZUR, XY). (52d) 
Next define 
8k = () 8°(QE. x,y) (53) 
OG x, E28 
and finally 
8° = () 8. (54) 
K 


Here c denotes complement with respect to the first octant So 2 0, 
S:2 0, S, 20. Thus, for example, $(Q¢x.vy) is a closed convex 
polyhedron bounded by seven planes. 

Note the similarity between (50) and (82) and between (81) which 
defines (P$x,y) and (52) which defines 8°(Q{%,x,y). Ror every dis- 
tribution in 9x, there is a distribution in @x that will give equality 
between the corresponding right-hand members of (31) and (52) when 
Z and U> are properly identified. We make this identification by 
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choosing IM = Mo, and by taking Z to be uniformly distributed over 
its M possible values. With a member of @x identified with each Q‘*i,x, 
in this way, we see that for this particular P$x, one has 


S(QUS.xy) = &(PRxy). 


Here the caret, *, denotes closure. Comparison of (34) and (53) then 
shows that Sx C ®x. Thus § C & or 


ROC Se (55) 
In Appendix F we establish 


Theorem 5: If R is an interior point of 8°, then for every K and every 
encoding C'x(R), 
P.(Cx(R)) 2 6 > 0, 


where 6 = 6(R) is independent of the encoding and of K. 


VI. SPECIFICATION OF THE CAPACITY REGION 


At the end of Section II, the capacity region was defined as the 
closure of the set of admissible rate points. Theorems 4 and 5 along 
with (55) show that C = & where & is defined by (81), (34), and (35). 
This characterization of © is of little computational value. It entails 
the calculation of the mutual informations appearing on the right of 
(31) for all distributions of form (32). A further infinite union over all 
values of K is then required. In this section we shall show how a much 
simpler description of @ can be obtained, one that is independent of K 
and hence much more suitable for numerical calculations. 

Central to the development of this simpler characterization of © is 


Theorem 6: The region © of admissible rates is convex. 


This theorem is proved by a time-sharing argument in Appendix G. 
By deleting words from a code, one obtains an additional obvious 
feature of the region © which we state as 


Theorem 7: Let REC. Then if 0 S Ry S Ra, a = 0, 1, 2, the rate 
vector R”’ zs also contained in ©. 


We return to our simpler characterization of ©. Let ®: denote the 
region specified by (81), (82), and (84) for K = 1. Since @ is the closure 
of ® as given by (35), @: C ©. From Theorem 6 it follows that also 


&’ = convex hull ®: Ce. (56) 


(The convex hull of a set @ consists of all points in @ and all points 
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on all line segments joining points of @.) We shall soon show that 
indeed &’ = €. 
As a step in this direction, in Appendix H we establish 


Lemma 4: For every distribution P§Qx,y(z, X1, X2, y) as in (32), 


K 

I(Xi; ¥|Z, X2) S mu T(Xu3 Y.|Z, X21), (57a) 
K 

I(X2; Y|Z, Xi) s du T(X243 Y,|Z, Xt); (57b) 
K 

I(X%1, X23 Y|Z) Ss >» T(Xu, Xe; Y:|Z), (57¢) 
K 

I(Xi, Xe; Y) s & (Xu, X2t5 Y;). (57d) 


Combined with (31) the lemma shows that 


A*(P#xy) 2 A(PR xy), (58) 
where &®*(P§¥x.y) is the set of rate vectors R for which 


K 

5 Bis YX VulZ, Xn, (59a) 
1X 

O0<kS K 2 I(Xo3 Y:|Z, X11), (59b) 

1K 
0Osht+hs K re T(Xu, Xoe3 Yr|Z), (59c) 
1 
Os R+Ri +k S K » (Xu, X 243 Y,), (59d) 


where the right sides of (59) are evaluated under Pix.y. Note that 
a*(P&xy), unlike ®(P¥£x.y), is a closed set by definition. 

Now, a typical term on the right of (59) depends only on the marginal 
distribution Pzx,,x.,v,(Z, V1, X21, yt). By summing (32) over the ap- 
propriate indexes and taking account of (26), it is seen that this 
marginal can be written 


P2xy,Xa¥ (2, Lit, Cat, Ys) 
= P72) Pxyyiz(t1t| 2) Pxeiz(@2e| 2) Py axiexee(Ye| Cre, Lat) 


which is a distribution of the form Px. for K = 1. Thus the right- 
hand sides of (59), which are the parameters defining the box-like 


MULTIPLE ACCESS CHANNELS 1057 


region ®*(Px,y), are averages of parameters that define the box-like 
region R(Pyiixny,),t = 1,2, ---, K. In Appendix I this fact and the 
convexity of the box-like regions are used to show that 


K 
convex hull U @(PRuxuy,.) D> A*(P#xy) (60) 
t=1 


from which it also follows that 


convex hull U @&(P&uxv)D U  § a*(P&xy,). (61) 
PY xr EP1 PY x, €0x 
We now have 


@’ = convex hull ®1 = convex hull closure U R(PHa x.y) 


PY x, EP 
> convex hull U R(PRixy) D U R*(P¥xy) 
PH x,E01 PRx,EPK 
> U- a@M(P&xy). (62) 
PPPs, €Ox 


Here the last inclusion follows from (58) and the next to last inclusion 
is (61). Using (84) and (62), we now see that ®’ D &x for every 
K. From (85) then @®’ D &, and since @’ is closed by definition, 
R’ D ® = ©. Combined with (56), this shows that ®’ = ©, and the 
formulation (10)—(12) is thereby established. 

It is to be noted that while this reduction permits calculation of C 
by evaluating mutual informations involving no more than four random 
variables, the size of the Z alphabet is unrestricted. In this connection, 
see the footnote in Section III. 

That we can indeed take the size of the Z alphabet to be 1 when 
computing the intersection of © with the plane Ro = 0, as claimed in 
Section III, is seen as follows. When Ry = 0, (11d) is weaker than 
(11c), since always I(X1, X2; Y) = I(X1,X2; Y|Z). Thus we need only 
consider (11a), (11b), and (11c) in defining regions ®(Pzx,x,y) in the 
R, — R, plane. But the right members of these equations are of the 
form 

T(X1; Y|X2, Z) = YL Pz(e)[(X1; Y|X2, Z = 2) 


(Xo; Y|X, Z) = ms Pz(z:)I(X23 Y|Xi, Z= 2i) 
1(X1, Xa; ¥|Z) = Z Pole) (Xi, X23 Y|Z = 2). 


An argument just like that of Appendix I now shows that ® € convex 
hull U; @; where &; is given by 0 S Ry S 1(X1; Y|X2, Z = 2), 
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0 < Re < (Xo; ¥ 1X4 Z= 2:),0 <s R, + R, < (Xi, Xo Y|Z = #). 
Each box-like region ®; can be thought of as obtained from a distribu- 
tion in which Z takes a single value with probability one. The formula- 


tion of Section III follows at once. 


VII. COMMENTARY 
7.1 Generalizations 
7.1.1 N Input Users 


The foregoing can be generalized to the case of a memoryless 
channel with N input users and a single output. The channel then 
is specified by alphabets Y, %1, X2, --- , Xw and transition probabilities 
p(y|x1, %2, -++, vy) foryE Y, a: € Xi, 7 = 1, 2, --- , N. Again we 
allow the information supplied to the input users to be correlated in a 
special way. 

We first write out the equations for N = 3 in full, and then indicate 
the general result. There are now seven independent sources, Si, S2, 
Ss, Siz, Sis, Ses, Si23 producing information at rates Ri, Ro, R3, Rie, 
Riz, Res, Riss respectively. There are three encoders. Encoder 1 sees 
the outputs of only Si, Siz, S13, Si2s; encoder 2 sees the outputs of 
only Se, S12, Ses, Si23; encoder 3 sees the outputs of only Ss, S13, Ses, 
Sie3. The decoder at the channel output attempts to reproduce sepa- 
rately the messages from the seven sources. Using block codes, for 
certain values of the rate vector R = (Ri, Re, Rs, Riz, Ris, Res, R123), 
the error probability of the system can be made arbitrarily small. The 
closure of the set of all such vector rates is called the capacity region @. 

€ can be found as follows. Let 


P123(2123), p12(212), p1s(Z13), P23(223) 
p1(21| 2128, 212; 213), P2(x2| 2123, #12, 223), ps(xs| 2128, 213; Z23) (63) 
be given probability distributions. Here z;C X:, 7 = 1, 2, 3. The 


Zi2, Z13, etc., have finite alphabets of unspecified size. We denote by 
P the distribution 


P= P123(2123) P12(2Z12) P13(213) P23 (223) p1( 1 | Z123, 212) 213) 
x P2(x2| 2123, 212) 223) D3(3| 2123, 213) Zo3) p(y | Xa, X2, £3). (64) 
Now let @(P) be the set of R such that 
0Oshs (Xi; Y | Zi, Z12, Z13, 223; Xa, X 3) 
Osk S (Xo; Y | Zins, Z12, Zi3, Z23, Xi, X3) 


0 < R3 S 1(X3; Y|Z123, Zz, Z1s, Z23, X1, X2) 
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Ri + Re S 1(X1, X03 Y|Z123, Z12, 213, Z23, X3) 


Ri + Rs S 1(X1, X33 Y|Z123, Zi2, Z13, Z23, Xe) 


Ro + R3 S [(X2, X33 Y | Z123, Z12, Z1s, Ze3, X1) 


0s 
0s 
0s 
0S Rhi+kh+ Ah S I(X, Xe, X33 Y |Z 13, Z12; Zi3, Z23) 
0s 
0s 
0s 


Rit+ Ro+ Rs + Re S (Xi, Xe, X33 Y|Zios, Zis, Z23) 


Ri+k+Rh:+ Ris S (Xi, Xo, X33 Y | Zi, Zr, Z23) 


Ri + Ro + Rs + Rog S [(X1, Xe, X33 Y|Z123, Z12, Z13) 


05 Rit Rot Rs+ Rio + Ris S 1(X1, Xe, X33 Y|Z123, Ze3) 


0S Rit Ro+ Rs + Rio + Res S 1(X1, X2, X33 Y|Z103, Z13) 


0S Rit Rot+ Rs + Ris + Ros S 1(X1, Xe, X33 Y|Z123, Z12) 


0S Rit Ro+ Rs + Ris + Ris + Ros S (Xi, Xo, X33 Y |Z) 


08 RFit Ro+ Re + Rie + Ris + Ros + Ries 
S (Xi, Xe, X33 Y), (65) 


where all the mutual informations here are computed with the distribu- 
tion (64). Let ®R =U R®(P) where the union is over all distributions of 
form (64) as the factors listed in (63) are varied. Then @ is the closure 
of the convex hull of &. 

The generalization to N users is simple in concept but awkward to 
describe. We do not dwell long on it here. There are now 2% — 1 
sources and @ is a region in a (2% — 1)-dimensional rate space. The list 
(63) is increased to contain 2” — N — 1 separate distributions for as 
many independent Z variables—Z12, Z13, +++ , Zea, +++ , Ziv3---v— 
and N distributions of form pi(ai| 212, 213, «++ , Z12---y), etc., where 
each z subscript contains the x subscript. Equations (64) and (65) are 
generalized in an obvious way. There are now DB (2 OD) se 1 ) equations 

= 
(65). © is given as the closure of the convex hull of the union of the re- 
gions defined by these equations. 

These results for N users were obtained by cursory examination of 
the rigorous proofs given in this paper for two users. As we have not 
had the courage to write out all the details, however, the assertions 
made for the N-user case must still be regarded as conjectures, or 
educated guesses. 


7.1.2 Continuous Amplitudes 


It would appear that our results can be extended in a natural way to 
channels with more general alphabet structures. For example, the 
channel might be specified by a conditional probability density 
P(y|a1, 2) where 21, 22, and y take all real values. Equation (11) 
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would remain the same, but the mutual informations are now given by 
integrals. Densities Pz(z), Px,iz(x1\z), Px.iz(v2|z) must be specified 
and the joint density of Z, Xi, Xe, and Y is the product (10) as before. 
Constraints, such as EX} = oj, EX = of must be imposed on these 
densities in taking the union indicated in (12). 

Again, we have not verified in detail the validity of the determination 
of © just given for continuous amplitudes. Caveat emptor. 


7.2 Some Problems 


Many research problems related to the subject of this paper remain 
to be examined. A brief description of some of these follows: 


(z) The footnote in Section III suggests that the size of the alphabet 
Z can be bounded in searching for the capacity of a particular channel. 
Is this conjecture true? 


(tt) The explicit construction of good codes for use on specific 
multiple access channels is an untapped field that leads to new problems 
not found on single-input, single-output channels. For example, even 
for noiseless channels (all channel probabilities zero or one) a coding 
problem exists since users compete with each other for the use of the 
channel. 


(iz) The region of rates for which error-free transmission with finite 
length codes is possible is not known. This region is analogous to the 
zero-error capacity of the single-input, single-output channel. 


(iv) For a particular multiple access channel it has been found that 
the region of admissible rates can be enlarged by allowing the encoders 
to observe the output via a feedback channel. This is in contrast to the 
situation for the single-input, single-output channel where feedback 
does not alter the capacity. In the multi-user case, however, a feedback 
channel increases the cooperation possible between the users and in 
general increases the forward capacity. How to calculate the region of 
admissible rates for multiple access channels with feedback is not 
known. 

(v) A special form has been assumed here for the correlation between 
the messages encoded by the two users. How does one handle more 
general correlations? Is the presently assumed form general in some 
asymptotic sense? 

(vt) Can one calculate the capacity region for some class of multiple 
access channels with memory? 

(vit) What is the rate distortion theory for these channels? 
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APPENDIX A 
Proof of Theorem 1 
Let Pes:.j,.4(C) be the probability of error when the source triplet 


(2, 7, k) is sent over the channel using coding C’. Let Pr (C) be the prob- 
ability of the particular coding C’. Then 


Paijn(N, Pox.) = L Pr (C)Pau,i.x(C), (66) 
where the sum is over all possible codings, that is, over all ways of 
choosing the code words Xiu, +++ , X1mpaty, X211, °° * » X2MoM,. But the 


right side of (66) can be interpreted as the probability of error in the 
joint experiment of drawing a code from the ensemble and transmitting 
(z, j, k) over the channel. With this interpretation in mind, we have 


Passo(N, PRx) = © Ps (67) 
where 
P, = Pr[U3 =i, Ui j, Ui = k|B] (68a) 
P, = Pr[U3 =i, Ut = j, UI # kB] (68b) 
P; = Pr[Uo = 2, Ui ¥ j, U2 ¥ k|@] (68c) 
P, = Pr [U3 # iB], (68d) 


where @ is the event {Up = 1, U1 = j, U2 = k}. We will find upper 
bounds for these four probabilities. 

We first compute an upper bound for P. Fix values for the N-vectors 
y, X1ij, Xeiz. Let z; denote the L-vector whose components 2:1, Zi2, «°° , 
Zi, were used in the choice of x1:; and Xei. Later we shall average over 
these quantities. 

Define @; as the event that 


PYYx.x0(y | Kriz, X2ix) 2 P¥kix.(y | X14, Kei) - (69) 


Note that the only random variable in this expression is Xi;;. Define 


PEW Gel2) = TT PSP Cal 20) (70) 
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where x is the N-vector obtained by concatenating the L K-dimensional 
vectors Xz, and z is an L-vector whose components are za, a = 1, 
2, ---, L. Then the probability of the event @; is 


Pr[@yj = 0’ Px? uy |Z), 
X1ij’ 
where the sum is over all values of x1:; satisfying 


PYYk.x(Y | X1i7, Xeez) 2 P¥k.x.(y | Xai, Xeen)- (71) 


Following Gallager,’ an upper bound to this expression is 





PY kx (y |X1ij-, Xocn) \% 
P @,1< PY’ (x17 Z; ( Y|XiX2 i's ) ; 72 
rly] 2» HB (Kass 124) PYYk.x,(y | X1sj, Xan) 2) 


for any s; 2 0. The summation is over all values of the N-vector X1;;’. 
For the same fixed values of y, X1:;, X2iz, and z:, let @ be the event 
that (69) holds for some value of 7’ not equal to 7. Then from Gallager® 


(page 136) 


i= 
Ai 


Prfe@]< ( > Pr tard)" (73) 


for any p; in the range 0 S pi S 1. Combining (72) and (73) we have 


Pr[@] s (Mi — Hal 5 PRP (X1| 21) 
PY kx (y |x; Xeik) ae 
+ Y¥|XiX2 } ; 74 
aan Xeik) ue) 


where the summation is over all N-vectors in (21). 
The probability of interest, P1, has an upper bound 


Pps YU YD Pk xy | X14, Xecn) PRP (X14 |Z 
y Xj xXaik 24 
X PRP (eux. |z)PP(z:) Pr [@], (75) 


where the inequality results from the fact that the occurrence of the 
event @ does not necessarily imply the event {U9 = 1, Ui ¥ j, U2 = k} 
but that the converse is true. Combining (74) and (75) and choosing 
8:1 = 1/(1 + p1), we obtain 


PrS (Mi — 1)" DUD PRP &2|z) PP) 


y x2 Zz 


x DY (PRP (112) P¥lkixs(y |X1, X2)) Ore +e, (76) 
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where the summations for y, X1, X2, and z are taken over all elements in 
the spaces (Y)%, (91)¥, (%2)%, and (3)# respectively. 

We note from (1b) that (4, — 1) < e¥*1, Now use the product 
form of (70) and write the right-hand side of (76) as an exponential of 
a logarithm. We find the desired result 


P, S exp {—N[Fi(m, Px.) — iki}, (77) 


where F; is given by (29a). The sums there are over all x1, Xo, y, and z 
contained respectively in (%1)*, (%2)*, (y)*, and 3. Reversing the role 
of U, and U2 one immediately obtains 


P» = €xp {—N(E2(p2, Px.) = polis) }, (78) 


where £» is given by (29b). 

The procedure for obtaining the upper bound for P3 is very similar 
to that used for P:. An outline of the proof follows. Fix y, x1:;, X2i., and 
z;. Define @ as the event 


PYkx.(y | X1iyv, Xen) 2 Prkx.(y | X11, X2ix) 
for some j’ # jandsomek’ #k. (79) 


It can then be shown that for any s; = 0 and0O S ps3 S 1 


Pr [@] S (Mi — 1)°°(M2 — Hal x E PRP Ga lz) 


x1 


PYkix.(y [x1 X2)_ \**] 08 
x PRI (x 23)( Ht te) | . (80 
X2|Z ( a| ) PY. x2 (Y |X1s, X2ik) 


Averaging over y, X1ij, X2iz, and Z;, and then setting ss; = 1/(1 + ps), 
we obtain 


Ps S (My — 1)**(M2 — 1)? YY) PP (z) 
Xx [x 2X PSP (x1 |Z) PRP (x2 |z) (PH k.x.(y |X1, X2))1/+e9 Jes, (81) 


Replace (Mi — 1)°3(Mz — 1)**by the upper bound e%#3(#1+2) | use (70) 
repeatedly, and write terms as exponentials of logarithms. One finds 


Ps S exp {—N[E3(ps, P&x.) — ps(Ri + Re) J}, (82) 


where £; is given by (29c). 

One minor change is made in the procedure to compute the upper 
bound for Py. We fix only the values of y, x1.;, and x2, (but not of z,). 
Define D as the event 


PLEA | Kriey, Kove) = PHY ¥X(y| X14; Xeon) (83) 
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for some 7’ #7 and any j’ and k’. Then for any s, 2 0,0 S ps S$ 1, 
Pr[ OD] S (Mo — 1)**(M1)?*(M2) 4 


(N; 8 

x [EE PRP, aa) coe ie |" 64) 
X1 X2 Py} xi(¥ | X13, X2ik) 

where 


PRP (X1, X2) = u PRR (z, X1, Xe). (85) 

Averaging over y, X1;, Xe: and setting s, = 1/(1 + p,), we obtain 
P, < (Mo sen 1)°4(M1)°4(Me ) ea 

x u [XD PRP x1, x2) (PHkix.(y |x1, X2) reo ]+e4, (86) 


X1 x2 


From (1b) we see that (Mo — 1) < e¥, An upper bound for M, 
follows from (1b) as 


M, <e@(Matj= eNR1(] + e-NR1) 
- exp {wm + EG Ee) 


zen {v| 


Using a similar upper bound for Me, we have that 


(M,) — 1)(M1)(M.) S exp 1 | Re +Ri+ RR. + TT |} (88) 





N 
From (88) and (70) we then obtain 
P, S exp {—N[E4(ps, P¥&,.) — ps(Ro + Ri + Re) J}, (89) 
where E, is given by (29d). Summing (77), (78), (82), and (89) results 
in (27) which was to be proved. 


APPENDIX B 
Proof of Theorem 3 
It can be easily verified that 


E.(pa, P&Qx,)|o-0 =0 for a =1,2,3,4. (90) 


It can also be shown by a straightforward but tedious calculation that 





5 Kis ¥ 1%, Z), Ae 

1 
aby) _ [RIES YI% 2) mez 
Ppa leno | 2 1K, Xs ¥|Z), aa8 

1 e7N (Bit Ra) 

x [ka Res Y) — —y_ a=4 
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where the /’s are mutual informations among K-vectors as computed 
under the joint distribution (82). Furthermore, from (29) it is seen that 
E, is analytic in pq in the neighborhood of p. = 0 and so can be ex- 
panded in a Taylor series about this point, a = 1, 2, 3, 4: 


E.lpa; Px.) 





[0+ 5 Lape + Out®); a2a=1,2,3 
=? 92 
l e—N (Rit Re) ( ) 
0+ | ele a — | + O00), a = 4, 


Here J, is the appropriate expression from (91) and O is the 
usual Bachmann-Landau order-of-magnitude symbol. Furthermore, if 
RC &(P&x.y), we have from (31) that 


1 


gie—Ro=6.>0, a =1,2,384, (93) 


Combining (92) and (93) with (80), we see that 


3 


P(Cy) S = exp {—Npalba + Oa(02)/pa]} 


eWN (Rit Re) 


+ exp {— Wow E aaa | creas 5 Ou(o)/o| (94) 


Now choose the integer L so large that 


e-LK (Rit Re) 


LK 


ba = 84 — 


is positive. Next, choose sufficiently small positive values of p1, p2, ps, p4 
so that 64 + O2(p2)/pa > 0, a = 1, 2, 3, and 54 + O4(p2)/ps > 0. The 
coefficient of N in each exponential of (94) is now negative, and we 
can increase N in multiples of K starting at N = KL until each term 
of (94) is less than ¢/4. Call this value of N, No = KL. Then (83) 
follows. Q.E.D. 


APPENDIX C 
Proof of Lemma 1 
By definition of P.(Cx) and H(Ui|Y, Uo, U2), 


Pa(Cx) = 2 Ld Pitas, J, y), (95) 


t jA#i*(y) 
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and 


H(U,|Y, Uo, U2) = 2X LL Pitti, J, k, y) 


a j &k 


lee rae 


By separating out the terms for which 7 = j*(y) in (96), one finds the 
identity 
T= A(U,4\Y, Uo, U2) = P.3(Cx) log (M, = 1) aaa h(P (Cx)) 

2 LV dk YL Pithuxt, j,k, y) 


a jAi*(y) k 
e Cx) 
l —PalCx) 
% 108 (ih, — DPA ovale By) 


th Ey Pa suix(t, J*(y), k, y) 


X log pa a, — F a(Cx)) (97) 


Ft vowrx(J*(y) |2, &, y) 
Now use the fact that log x S$ x — 1 to obtain 


Pal its r(% k, y) + 


Beit ia ie 


= Pou,u,v.x(t; 7*(y), k, y)] 
= Pa+ (1 — Pa) -1=0. (98) 


Replacing M, by M,+ 1 yields (47a). Equations (47b), (47c), and 
(47d) are proved in a similar way starting from the definitions 


Peo(Cx) = x Ps pS ey) Pvt, i; k, y); (99a) 
t fj 
Pa(Cx) = Dd Ds Pittivx(t, J, ky), (99b) 
y t (jk) AG*(y) RR (y)) 
and a 
P.(Cx) as = a oS : P&o.vex (1, J) k, y). (99c) 
y (t,9,k) AC, I*,K*) 
APPENDIX D 
Proof of Lemma 2 


For part (a), we write a complicated conditional mutual information 
in two different ways: 
I(X1, U1; Y| Us, Xe, Uo) 
= I(Xi; Y| U2, Xo, Uo) + I(U,; Y| Xi, U2, Xe, Uo) 
= I(U1; Y¥| U2, X2, Uo) + T(K1; Y| U1, U2, Xe, Uo). (100) 
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Now 
I(Xi; Y| Us, U2, X2) 

=f {log PProovivix.x:(Y | Us, Ui, 02, Xi, x | 

P¥Poux.[¥ | Us, Us, X») 

PY x,(¥ |X, X») 
PH rox, (¥ | Us, X2) 
where the equalities result from the special form of the joint distribu- 
tions as given by (39) and (40). For the next mutual information in 
(100), we have 
(Vi; Y| Us, U2, Xi, X2) 


= E { log F itwwrwenxa(¥ | Uo, Us, Un Xs, Xs) i 
P¥Poux.x(¥ | Uo, U2, Xi, X2) 


2 B | log \ = 1(X1;¥|Uo, X:), (101) 


PYik.x.(¥ |X1, Ke) 
P¥ik.x,(¥ |X1, Xe) 


The third mutual information in (100) can be written 


I(U,; Y| Vo, Ua, X2) 


= B log i =0. (102) 


PS (YoU Ua: Xs) 
= By Y| UoUi1U2X2 D » V2; 
[8 PRvoxs(¥ | Uo, Us, Xs) 
P¥ftwuiv.(¥ | Uo, U1, U2) 
= 0U102 5] ) _ I ay ' . 1 
B {log P&RvAY|Uo, Us) Ce eras Ne 


Finally, 
I(Xi; Y| Uo, Ui, 0», X2) = 0, (104) 


since all mutual informations are non-negative. Combining (100)- 
(104), we obtain (48a) which completes the proof of part (a). 
The proofs for parts (b), (c), and (d) follow in a similar manner. 


The equations corresponding to (100) are: 
Part (b), 


I(U2, X23 Y| Uo, U1, X1) 

= I (Xa; Y| Us, Ui, X:) + I(U2; » a Uo, 0, Xi, X2) 

= I(U 2; Y| Us, Ui, Xi) + I (X23 Y| Us, U1, U2, X1); (105) 
Part (c), 
(Ui, Up, Xi, Xo; Y| Us) 


= I(K1, X23; Y| Uo) + (U1, U2; Y| Uo, X1, Xe) 
te I(U,, U2; Y| Uo) ae T(X, Xe; Y| Uo, U1, U2); (106) 
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Part (d), 
(Uo, U1, Us, Xs, X23 Y) 
= I(Xi, Xo; Y) + (Uo, Ui, Ud; Y¥ |X, Xp) 


= I(Uo, Ui, U2; Y) + I(X%, Xo; Y| Uo, Ui, U2). (107) 
Q.E.D. 


APPENDIX BE 
Proof of Lemma 8 
We use the identities: 

(a), (Ui; Y| Uo, U2) = H(U,| Uo, U2) a H(U,| Us, Ui, Y); (108a) 
(b), I(U2; ¥|Us, U1) = H(U2| Uo, U1) — H(U2| Uo, Ui, Y);  (108b) 
(ec), I(U,, Us; Y| Uo) = H(U,, U2| Uo) _ H(U,, U2| Uo, Y¥); (108c) 
(d), I( Us, Ui, U2; Y) = H(U,, Ui, U2) = H(U,, Ui, U»|Y) . (108d) 
From the joint distributions of the random variables Us, Ui, and U2 
given in (2), we have: 
(a), H(U1|Uo, U1) = H(01) = KR;; (109a) 
(b), H(U2|Uo, U1) = H(U2) = KR; (109b) 
(c), H(Ui, U2|U0) = H(U,, U2) 

= H(Ui) + H(U:) = K(Ry + R2); (109¢) 
(d), H(Uo, U1, U2) = H(Uo) + H(U1) 

+ H(U2) = K(Ro + Ri + Rz). (109d) 


Combining the appropriate equations in (108), (109), (41), (47), and 
(48), we have (49) which was to be proved. 


APPENDIX F 
Proof of Theorem 6 

If R is an interior point of S*, then there is a sphere, o, of radius 
n(R) > 0, centered on R such that every point in o is also in $*. Thus 


every point in $8(Q%.x.,vy) must be distant more than 7(R) away from 
R, and this is true for every K, and every Q{ix,x, in 9x. This in turn 
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implies that one of the inequalities 


R= 7 UKs Y| Uo, X») > 7(R) 


1 
R, — x 1 (Kes ¥| Uo, Xi) > 7(R) 
1 (110) 
Ry + Ro os K ks Xo; Y| Uo) > n(R) 


] 
Ro + Rit Re. - Feoee X2; ¥) > (R) 


must hold for every encoding C'x(R) of the sort under consideration, 
whenever R is interior to 8°. 

Now from (41), R, = R., a = 0, 1, 2, so that one of (110) holds also 
when the R’s are replaced by R”s. From Lemma 3 we then find that 


RPea(Cx(R)) + 7% h(Pea(Cx(R))) > a(R) 


for at least onea, a = 1, 2,3,4, (111) 
where we define 
P.4(Cx(R)) = P.(Cx(R)) (112) 


and for any rate vector R we define an associated 4-vector R by 
(Ri, Re, Rs, Rs) = (Ri, Re, Rit Re, Ro + Rit Re). (118) 
But from (87) and (88) we see that 


A 


ek Ra 


K 





R,< Ra + < (R. + e-*s), (114) 


so that 
RiPea(Cx(R)) + ME eal) 

S (Ra + €**)P.a(Cx(R)) + h(Pea(Cx(R))). (115) 
Combining (111) and (115) we find that 


(Ra + e-#+)Pee(Cx(R)) + h(Pea(Cx(R)) = n(R) 


for at least onea, a =1,2,3,4. (116) 
Now 


h(x) S$ 2vx, OS2xS1, (117) 
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as can be seen by the following simple argument. From 
O<albe d= 2)*], 
it follows that 2Z <1+2Z2-+ Z?/2 S e#,forZ = 0. But for Z < 0 we 


also clearly have 2Z < e% so that 2Z < ee” for all Z. Substitute 
Z = log v(1 — £)/t to obtain 
1-—t -vl-t 


log —— < Ze 


ey (ee 
t Vi Vi 


or 





2 1—t = dt 
[08 ; a< foo, eos 1 


Perform the integration and take the limit as e > 0. Equation (117) 
results. 

Use (117) in (116) to find that (R, + e-**)P., + 2VP.. = 7(R) 
for at least one a, a = 1, 2, 3, 4. This implies that 


V1 + n(R)[Ra + em Fe] — 172 
Pr(Cx(R)) 2 | —F MEL I | = () > 0. 
Since P,(Cx(R)) = max«[P-a(Cx(R))], we find finally that 
P.(Cx(R)) 2 6&(R) > 0, (118) 


where 6(R) = min, 6.(R) is independent of K and the encoding 
Cx(R). Q.E.D. 


APPENDIX G 
Proof of Theorem 6 

We first show that for every positive integer n and every integer r 
such that 0 Sr S n, 


bs = "RER. (119) 





REG, RERSR="Ri + 


Since @ is the closure of ®, and since the rationals are dense in the 
reals, (119) implies that if Ri C © and R. € @, then for every \, 
05) 81,Rs; = AR, + (1 — AJRz € @, which shows € to be convex. 

To establish (119), we use the notion of time sharing to generate 
new codings from old ones. Suppose we have two codings C'y(Ri) and 
Cy(Re) both of block length N and with numbers of words M, and M, 
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respectively, where as usual 


MD. — (M oa, M120, Mo.) = (PeN Roel PeN Rial TN Roa I) 


120 
a =1, 2. a20) 


Denote by P.1 and P.2 the respective error probabilities achievable 
with Cy(R:) and Cy(R:2). Now consider the possible channel input 
vectors that can be obtained by using Cy(R:) r times followed by 
(n — r) uses of Cy(R2). The totality of these input vectors, each of 
nN components, can be thought of as the words of a new code of block 
length nN. Denoting its word size parameter by M, we have 


M;= (M i1)"(M 2)", i= 0, it; 2. (121) 


If we use the decoders for C'y(R:) and Cy(R:2) to decode the appropriate 
blocks of length N in this new larger code, the error probability for 
the new code, P., will satisfy 


1—-P,=(1— Pa)(1 — Pe)" = 1 —rPadfl — (n — vr) Pee] 
>1-—rPa— (n—1r)Pe 
so that 
Pee th ee iS HL e: (122) 


Here we have used the fact that the channel is memoryless. 

We now use this time-sharing notion to establish (119). Suppose that 
integers n and r are given with n > 0,0 Sr S n and that R; and Rz 
are rate points in ®. Suppose further that « > 0 is given. Then, from 
Theorem 4, there exist positive integers K, and L, and a sequence of 
codings Cy,(R1), Ni = Kili, Ki(Zi+ 1), Ki(Zi + 2), «++ such that 
for each coding of the sequence P.(Cy,(Ri)) > ¢/n. Similarly there 
exist integers K, and L. and a second sequence of codings C'y,(R2), 
Ne = Kelo, Ko(L2 + 1), Ke(Le + 2), «++ such that for each coding in 
the sequence P.(Cy,(R2)) < e/n. We now choose one coding out of 
each of these sequences of codings in such a way that they are of the 
same block length N. A suitable choice for N is the least common 
multiple of KiL; and K2L». Call the two codings Cy(R:) and C'y(R2). 
Their error probabilities are Pea < e/n and P.2. < e/n. Time sharing 
them as discussed earlier yields a new coding C, of block length 
Nz; = nN, code book size M given by (121) and (120), and error 
probability 

P.srPat(n—-r)Pea =ro+ (n — 1) 


é€ 
—=€ 
nr 
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from (122). Now from the fact that Fa Uy! = Taxy1, (121) and (120) 
give 
M, = VeN Rall eNRialn—r > PeN Ir Rat (n—r) Ria) 7 

= VeNsRis 1, =O, 1,2, 
where R; is as in (119). Thus by deleting some words from the code C 


we can obtain a coding with rate R;, and block length N3, that has 
error probability P. << «. Q.E.D. 


APPENDIX H 

Proof of Lemma 4 
Consider (57a). We write 

(Xi; Y|Z, Xz) 

P¥irxx.(¥ |Z, X1, Xe) 
P¥irx,(Y |Z, Xe) 


K 
i Pyix,x,(Y:| Xu, X 24) 


= E log 


= FH log 


Shee ee eS ee te ee pk 
i PHOaxevr vial Y |Z, Xa, Y1, uty Y 1-1) 
K 
= a [LH(Y.|Z, Xe, Yo, +++, Yin) — H(Vs| Xi Xa) ]. (128) 
Here, fort = 1, the conditioning on Y,, --- , Yi:_-1is to be omitted. But 
H(Y,|Z, Xo, Ys, ---Yi-1) S H(Y¥1|Z, Xo), (124) 


since removing conditioning random variables cannot decrease an 
entropy. Combining (123) and (124) we have 


K 
T(%1; Y|Z, X2) S 2 CH(Y.|Z, Xo) — H(Vi| Xu, Xe) ], (125) 
or 


K 
T(&1s ¥|Z, Xs) SV U(X Ye|Z, Xe). (126) 
t= 


The proofs for (57b), (57c), and (57d) are similar. 


APPENDIX I 
Let numbers A;, B:, C:, and D; be given that satisfy the inequalities 
0OSA,5C, (127a) 
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OSB, SC, (127b) 
0 s C, < At + Bi, (127c) 
0 < Cc; < Di, t = 1, a; ae ss K. (127d) 


Let &; denote the set of points (x, y, z) in three-space such that 


Ose A,, (128a) 
OsySB,, (128b) 
Osa+ySC, (128c) 
Ose+tyt+2z2s D, (128d) 
fort = 1, 2, ---, K. A sketch of ®; is shown in Fig. 7, corresponding 


to the case in which all the inequalities in (127) are strict. We further 
define 


t=1 


Now consider the region ®p» consisting of all points (2, y, z) such that 


K 
jee odes ss A, (130a) 





Fig. 7—The convex region ®:. 
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K 
0SySBR=—>B, (130) 

K i= 

1X 
Oset+ysQO=z25 D.C, (130c) 

=1 

1x 
OStt+y+zSD=R2 Dz (130d) 

t=1 

Our first goal in this appendix is to show that 

®o € convex hull ®. (131) 


By summing the inequalities (127) and using the definitions of Ao, 
Bo, Co, and Do given in (130), we see that (127) also holds for t = 0. 
®o, too, then has the form shown in Fig. 7. As is seen, each region 
R., t = 0, 1, --- , K, is convex and has ten extreme points, the com- 
ponents of which are listed below: 


To; = (0, 0, 0) 

Tir = (Az, 0, 0) 

To, = (Ai, C; — Az, 0) 

r3, = (C; — B, Bi, 0) 

rae = (0, Bi, 0) 

Y51 = (Ai, 0, Di — At) 

te: = (Ai, Cy — At, Dt — C2) 
tr, = (C; — B., Bi, Di — Ci) 
rs, = (0, B:, D: — Be) 

To, = (0, 0, Dy). 


(132) 


[Some of these points may coincide if there are equalities in (127) 
instead of strict inequalities. ] For the extreme point of Ry we also have 


1 & ‘ 
fio => Dd Tit, 7=0,1,--:,9 (133) 
K 1 


which follows directly from (132) and the definitions on the right of 
(130). We recall that a convex body is characterized by its extreme 


MULTIPLE ACCESS CHANNELS 1075 


points: r © ®; if and only if 


9 
r= > AT ity (134) 
i=0 
where 
9 
420, ¢=0,1,---,9 and DA; =1, ¢=0,1,---,9. (135) 
0 


Equation (131) is now easy to establish. It is clear that the convex 
hull of ® is the set of all points that can be written in the form 


9 s«K 
= > > Ui it, (136) 


2. 
o 
oo 
ll 
~ 


where 


K 
We 20, 6=0,1,-+°,9 t=1,---,K, Y Yue =1. (37) 


iM 


Now let r be any element in ®. Then r can be written in the form 
(134)—-(135) with ¢ = 0. Substituting from (133) yields 


9 Kj, 

=> vo sri. (138) 
‘m0 121 K 

But defining 


) 4 = 0, o+2 9) be Lye glk, (139) 


PA 2° 


. — 
Uy = 


we see that u;, = 0 and 


> yu, =a (140) 


Comparison with (136) now shows that r is in the convex hull of @. 
Equation (131) then follows. 


The application of the foregoing to (60) is immediate. Let 
A, = [(Xu3 Y:| Xe, Z) 
Be= T(Xat3 Y.|Xu, Z) 
C, = L(Xu, X23 Y,[{Z) 
D, = [(Xu, Xor3 Yu) 


= 1, --- , 9. Equations (127) are satisfied. We then identify &, of 
this appendix with ®(Pzx,,x,,y,) of (60),¢ = 1,2, --- , K, and ®) with 
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a*(P&x,v) of (59) which is consistent with (130). Then (129) and 
(131) yield (60). Q.E.D. 
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A Compromise Equalizer Design 
Incorporating 
Performance Invariance 


By F. J. BROPHY, G. J. FOSCHINI, and R. D. GITLIN 
(Manuscript received February 20, 1973) 


We give a solution to the problem of designing a fixed compromise 
equalizer for use in transmission systems involving an ensemble of random 
channels. The signal and noise spectra, along with the second-order 
statistics of the channel ensemble, are used to find the equalizer charac- 
teristic that minimizes the mean-square distortion between the equalizer 
output and a scaled version of the transmitter output. The key departure 
from previous work is that the criterion better captures practical per- 
formance invariance; specifically, the cost functton incorporates the 
insensitivity of a well-designed demodulator to any amplitude scaling or 
time delay introduced by a particular channel. After demonstrating that 
the optimum equalizer shape is related to the principal eigenfunction of a 
normalized channel correlation function, we consider several special cases 
that give further insight into the properties of the solution. We find that 
the equalizer amplitude is attenuated over those frequencies where the 
stgnal-to-noise or signal-to-channel-variance ratios are small. The analysis 
confirms the standard engineering practice of inverting the average channel 
in the absence of noise and when the variance of the channel characteristics 
ts small. 


I. INTRODUCTION 


A fixed compromise equalizer is frequently employed in data trans- 
mission systems to compensate for linear distortion introduced by a 
channel drawn from a random ensemble.! Typically, compromise 
equalizers find application in systems which, because of economic or 
other considerations, do not use an adaptive equalizer. It is possible 
that, even when adaptive equalization is used to compensate for a 
particular channel characteristic, one might use a compromise equalizer 
to provide a good initial channel and thereby reduce the receiver 

1077 
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adaptation time. As its name suggests, the equalizer is a fixed linear 
filter that effects a compromise by compensating for an ‘‘average”’ 
channel. We propose a procedure that uses the statistics of the channel 
ensemble, the modulated signal, and the additive noise at the demodu- 
lator to design a filter that minimizes a performance measure ap- 
propriate for most transmission systems. The performance measure is 
an adaptively scaled mean-square error. For example, it is particularly 
well suited for use in a data transmission system and results in a filter 
that is significantly different from that obtained by directly minimizing 
the mean-square error.? 

In order to avoid being restricted by nonlinear demodulation tech- 
niques (e.g., those used in FSK or DPSK data systems) and to be 
able to accommodate asynchronous signalling, the equalizer operates 
directly on the received passband signal and is thus a channel, rather 
than a synchronous, equalizer. Our performance criterion is the con- 
tinuous time mean-square error between the equalizer output and an 
adaptively scaled version of the transmitter output. This scaling needs 
to be done only once in the design of the filter and not each time a new 
channel is dialed up. The scaling is such that the filter is invariant to 
any amplitude scaling, sign inversion, or time delay encountered in 
transmission over a particular channel; this type of invariance is 
appropriate for a compromise equalizer used in most transmission 
systems, since amplitude scaling and a fixed time delay will not be 
“seen” by a well-designed demodulator. 

Since the criterion is quadratic in nature, the optimum filter re- 
sponse is determined by the second-order statistics of the signal, noise, 
and channel ensemble. The best filter shape is shown to be the principal 
eigenfunction of an integral operator whose kernel is a weighted channel 
correlation function. An explicit design procedure is described in the 
text, and several interesting questions are discussed as well. Some of 
these questions are: 


(t) How different is the compromise equalizer from the inverse of 
the average channel characteristic? 

(27) Suppose there is no amplitude distortion but only delay distor- 
tion. What is the nature of the compensation? Is there amplitude 
as well as delay compensation? 

(zit) How sensitive is the filter design to different signal spectra? 


In Section II the compromise equalization problem is formulated, 
and the distortion measure is discussed in considerable detail. The 
determination of the optimum filter is described in Section III, and 
an example is provided in Section IV illustrating the design technique. 
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CHANNEL ee X(t) | bemop- 
{nit)} g(t) ULATOR 


Fig. 1—Transmission system with a compromise equalizer. 
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n(t) 


Il. PROBLEM FORMULATION 
2.1 System Configuration 


We begin by specifying the framework of our discussion. To accom- 
modate a system employing nonlinear demodulation, the compromise 
equalizer is assumed to be placed in the passband. Since we also want 
to provide a setting general enough to include asynchronous transmis- 
sion, no restriction is placed on the form of the filter other than require- 
ment of a finite energy impulse response. As shown in Fig. 1, s(t) 
denotes the modulated signal, n(¢) the received additive noise, x(t) the 
compromise equalizer output, and g(t) the equalizer impulse response 
(which we ultimately seek to specify). The ensemble of channel impulse 
responses, {h(t)}, as well as n(¢) and s(t), are all assumed to be con- 
tinuous in quadratic mean and statistically independent of each other. 
Both s(#) and n(é) will be taken to be zero mean, to be wide-sense 
stationary with finite power and to possess power spectral densities 
S(w) and N(w), respectively. To make the subsequent analysis precise, 
we assume h(t) has bounded absolutely integrable sample paths (with 
probability one); thus, the ensemble of channel frequency charac- 
teristics {H(w)} is well defined.t 

Our problem is first to select an appropriate optimization criterion 
and then to choose the compromise equalizer that minimizes this 
measure. To avoid the specifics of particular demodulators, we will be 
concerned with preserving the fidelity between s(t) and x(¢). A natural 
first choice for a distortion measure is the continuous-time mean- 
square error 


& = (Lz) — s(t — 7) ?), (1) 


where ({ ) denotes the average with respect to the signal, noise, and 
channel ensemble, and 7 is an arbitrary delay. Using this measure, 
Maurer and Franks? found that the optimum filter is given by 


S(w) (Hw) )*e77"" 


Ske) = SNE) ON)? 


(2) 


t The process {H(w)} is assumed to have a square integrable covariance, and the 
variance is assumed to be nonzero for the frequency range of interest. 
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where the asterisk denotes the complex conjugate and H(w) is a member 
of the channel ensemble. It is easy to see that G,(w), given by (2), is 
not just the Wiener filter for the average channel, since the mean- 
square channel dispersion (|H(w)|*), rather than |(H(w))|?, modifies 
the filter characteristic. We note that, even as N(w) — 0, the filter does 
not generally invert the average channel. This will be the case when 
the channel variance is extremely small so that (| H(w) |?) & | (H(w))|*; 
this would occur, for instance, when the ensemble consists of only one 
characteristic. 

Care must be exercised in determining how to use the design tech- 
nique that results in the filter given by (2). For example, suppose the 
individual transfer characteristics are greatly varied in the degree of 
attenuation they impart across the band of interest. It can follow 
(depending on how probabilities are attached) that the low-loss chan- 
nels determine the character of the averages in (2)—the ‘“‘whomper”’ 
effect. Hence, linear distortion in the lossy channels may not be 
suitably equalized. However, if linear distortion is uniformly the 
dominant impairment, the noise level can be set to zero and the 
‘“whomper”’ effect easily eliminated by preparing the data by normaliz- 
ing the channels prior to averaging. For example, the normalization can 
be accomplished by individually scaling the characteristics so they 
have the same energy in response to some pulse or so they have the 
same gain at some central frequency. Even if linear distortion is not 
the uniformly dominant impairment, there may be applications where 
the noise is set to zero and a normalization of the channels made prior 
to computing the optimum equalizer. In such a procedure, one is trad- 
ing noise immunity for immunity to linear distortion. Another case of 
importance in applications is when the channels have the same ampli- 
tude characteristic but different phase characteristics; here, of course, 
the ‘‘whomper”’ effect is nonexistent. The considerations of this para- 
graph will also apply to the design technique we shall develop in the 
following sections. 

While the & criterion provides an interesting and tractable formula- 
tion, it has the shortcoming that it understates the capability of most 
demodulators. A striking example of this occurs if the ensemble {h(¢)} 
has zero mean. The zero mean is reasonable for systems subject to 
occasional “phase hits,” the mathematical implication being that, if 
h(t) is a possible channel, then —/h(#) is just as probable. Then G,(w) =0. 
Yet, in practice, one would expect to do better than “‘pull out the plug 
and go home.” 

At the other extreme we could use an information theoretic type 
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criterion. For example, we could choose G(w) to maximize the average 
mutual information between z(é) and s(¢). This criterion is mathe- 
matically tractable; in fact, the solution is trivial—G(w) can be any 
characteristic which is nonzero over the same frequency range as 
S(w).t The shortcoming of this criterion is (as the solution suggests) 
that it overestimates the demodulation capability. 

As the state of the art in demodulation advances, we would antici- 
pate an evolution of criteria away from the mean-square error toward 
the information theoretic. Clearly, a good criterion should give the 
demodulator credit for what it can realistically accomplish and at the 
same time pose a tractable optimization problem for determining G(w). 

With this motivation, we propose a mean-square-error criterion 
which reflects the fact that the performance of a well-designed demodu- 
lator is insensitive to scaling of the input (the automatic gain control 
feature), an input sign change (the information is generally differ- 
entially encoded), and, of course, a time delay. Under such a criterion, 
the compromise equalizer will no longer be implicitly constrained by 
attempting to faithfully reproduce the modulated signal. 


2.2 An Adaptively Scaled Mean-Square Error 


Based upon the above discussion, we consider the following adapt- 
ively scaled mean-square error 


é = min (Lx() — As(t — B)]?)s,n, (3) 


where A and B are real numbers and the averaging is over the signal 
and noise statistics with both the channel and equalizer held fixed. 
The criterion is meaningful under the assumption that the signal-to- 
noise ratio at the receiver does not change appreciably from channel 
to channel. We stress that the considerations mentioned in the third 
paragraph of Section 2.1 apply here, as well. The optimum equalizer 
is obtained by averaging & over the channel ensemble and then 
minimizing the result with respect to the equalizer transfer function 
G(w), subject to a power constraint on the demodulator input. The 
quantities A and B, which are channel-dependent, provide an adap- 
tively scaled reference signal As(¢ — B). The reference is adaptive in 
that, for each realization of the channel, A and B are chosen to mini- 
mize &. Notice, for example, that, if a particular channel introduces a 
sign inversion, then A = —1 will remove this effect. When a channel 


t This comes about because no information is lost when the signal is subject to a 
reversible operation (such as a channel). 
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has a more complicated phase and/or gain characteristic, it is no 
longer apparent what the optimal value of A should be; however, we 
shall shortly see that the minimizing value of A can be determined 
analytically. We shall also consider the determination of B. Thus, the 
filter will not expend any of its degrees of freedom by attempting to 
compensate for a sign inversion, amplitude scaling, or time delay 
introduced during transmission. It should be clear that, since A and 
B depend on the channel characteristics, they are random variables. 
Simply put, the criterion given by (8) forces the equalizer to minimize 
only the portion of the output signal that does not look like a scaled or 
delayed version of the transmitted signal. 

We now consider the properties of the adaptively scaled mean- 
square error. We begin by letting 


= ((x(t) = As(t rac BY Pan; (4) 
L.e., 
= min & 


Carrying out the indicated average gives 


T= [7 (S@[IFO) |? — 24e**F@) + 44) + |6@[NWIS, © 


where S(w) is the power spectral density of s(t) and F(w) is the product 
of G(w) and H(w). (Notice J does not change when F'(w) is replaced by 
Re{F(w)}.) To find the minimum of J with respect to A and B, we set 
to zero the partial derivatives of J with respect to these variables and 
find that 


al = 45 des i dey _ 
Sa =~ 2 |, Slole**Fw) 5° +24 [ Slo) 5° = 0 (6a) 
— ng i jwS (w) e%2F (w) = 0. (6b) 


From (6a) we have Aopt given by the correlation ratio 


20 ie dw 
[/ Seye*F) 3 _ (s(t — B)x(t)) 


Big es a 
[soe (s*(¢ — B)) 


(7) 


The interpretation of Bop: is facilitated by letting 


yt) =f joF Swe 52, (8) 


COMPROMISE EQUALIZER DESIGN 1083 


R(t) d 
F = y(t) —>B 


t=Bopr 


R(t 


Fig. 2—Interpretation of optimum delay (Bopt) and amplitude (Aopt) scaling. 
Bopt is one of the instants when y(t) is zero. 


which is recognized as the response of a linear filter with transfer 
function jwF(w) [i.e., F(w) followed by a differentiator] to the input 
R(t), where R(é) is the (inverse) Fourier transform of S(w) and is the 
signal correlation function. Comparing (6b) and (8), we see that Bop: 
is one of the instants when y(t) is zero. We illustrate this interpretation 
of Bopt In Fig. 2, where we also indicate how Aop: may be obtained in 
a similar manner. Determining Bo»; is a very difficult problem, since 
it is tantamount to asking for the zero crossing of a signal from knowl- 
edge of its Fourier transform. In order to proceed further, we will 
approximate Bop: by the delay at midband, which we conveniently take 
to be zero for each channel. 

Using tlie value of A.p: given by (7) and setting B = 0, we have 


be : dw 
F -[" |@() |*18(w) |H@) |? + N@)IF 


ss | i : S(w) GW) Hw) SF), (9) 


where 
je f i S(w) we. 


The optimum filter is obtained by first averaging (9) with respect to 
the ensemble of channels and then minimizing this average with 
respect to G(w).' Before doing this, we give geometric interpretations 
to both the criterion and the optimum filter. 


2.3 A Geometric Interpretation of the Problem 


Finding a geometric framework in which to view the optimization 
problem will be quite helpful in understanding the nature of the 
solution. To put the problem at hand in such a setting, we introduce 


t The minimization is done subject to a constraint on the average output power. 
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1 





GH, 


Fig. 3—Geometric interpretation of criterion. The adaptively scaled mean-square 
error may be interpreted as the average squared distance from the point GH (which 
represents an equalized channel) to the ray 1 (which represerits a distortionless 
channel). 


the signal-weighted inner product 


(U,V) = [" UW)V*W)S&) S, (10) 


where the vectors U and V represent the functions U(w) and V(w) 
respectively, and the vector 1 will correspond to the real function of 
unit amplitude. Suppose that the noise is set to zero; then, in terms of 
the above notation, we can writet 


§ = ||GH|? — + (GH, 1), (11) 


where the vector GH corresponds to the function G(w)H(w). For 
convenience we set the signal power equal to unity, ie., a = 1, and 
apply the Schwarz inequality, which gives 


& = ||GH||? — (GH, 1)? 2 ||GH||? — |[GH||?-|[1]/? = 0, (12) 


where the lower bound is achieved when GH is proportional to 1. 
From (11) we have 


6 = (GH, GH) — (GH, 1)? = (GH, GH)| 1 a (GH. GH) | 


= ||GH|/?[1 — cos? @] = [||GH|| sin 6)’, (13) 


t The norm of the vector U, denoted by ||U||, is given by (U, U 


+ Thus, if we have only one channel characteristic and no noise, tie equalizer will 
invert the channel. 
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where 6 is the angle between GH and 1. Equation (138) provides a very 
useful interpretation of the error 6. As shown in Fig. 3, ||GH|| sin @ is 
the distance from the vector GH to the ray colinear with the vector 1. 
Hence, the average of & with respect to the channel ensemble, which we 
denote by (&)u, is the average squared distance from GH to the ray 1. 
Thus, for a given channel-equalizer constellation, {GH;}, the equalizer 
is chosen to minimize the dispersion about the ray 1. 

In order to get a feeling for the capability of the equalizer to modify 
the channel constellation {H;}, we associate with G(w) a linear operator 
G that maps a particular channel H; into GH;; we write this operation 
symbolically as 

G: H; — GH,. (14) 


The equalizer operator, GS, is called diagonal since it modifies H(w) in 
a pointwise fashion to produce G(w)H(w). Suppose, for the purpose of 
illustration, we relax our hypothesis and assume that the channel 
ensemble has energy only at two values of w, w; and w.. In this case, the 
operator G is particularly simple since the point H = (hi, he) is mapped 
into the point GH = (gihi, goh2), where h; = H(w:) and g; = G(w:). 
The locus of points (g1h1, goh2), subject to the average power constraint 
on the demodulator input x(t) 


giki + gak. = 1 (ki = E{|H(w:) |?S(@:)}), (15) 


describes the manner in which the equalizer redistributes the channel 
ensemble so as to minimize the average squared distance to the ray 1. 
By noting that the point (g1/1, gehe) satisfies the relation 


(gih1) "ky (g2hz) "ke =| 
(Ai)? (h2)? 


we see that the locus of the points (g1h1, g2h2) subject to (15) is an ellipse 
centered around the origin whose major and minor axes are parallel to 
the a-y axes and are of length |hi/kj| and |h2/k§|. Thus, the effect 
of the equalizer on the channel array, as shown in Fig. 4, is to allow 
each equalized channel, GH; = {gh{, gh$?}, to move on an elliptical 
surface. The nature of the compromise by which the optimum filter 
shape is chosen should now be clear. Each channel wants to move as 
close to the unit ray as possible—this will usually lead to conflicting 
requirements, i.e., as one channel is brought closer to the unit ray, 
other channels will move further away from this ray. For a multiplicity 
of channels there will be an “elliptical flow’? along the respective 
ellipses which terminates when the best filter shape has been found. 





te (16) 
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Fig. 4—Geometric interpretation of the effect of the compromise equalizer on the 
bina ensemble. A channel, represented by the point H,, is modified by the equalizer 
to give the equalized channel GHj. Fs 


The above interpretation would of course be valid in a Hilbert space 
of dimension large enough to accurately represent the function H(w). 


III, DETERMINING THE OPTIMUM EQUALIZER CHARACTERISTIC 
3.1 Analytic Solution 


Having developed a geometric representation as an aid to under- 
standing the equalizer design problem, we are now in a position to 
explicitly determine the best filter shape. Returning to (9) and averag- 
ing with respect to the channel ensemble, we have 


(8)a = [~ (Ge) [*18) Halo) + NW)IS 


-2( [ s@soerwenu@wHom 2, (7) 
where 
Hrms(w) = V(|H(w) |*). (18) 
If we let 
Q(w) = VS(w)Hims(w) + N(w)G) (19)" 
and introduce the weighted channel covariance kernel 
ead 8(w)S0)(H*()H0)) ip 


: EGE + N(w) JLS@) Hine) + NO)’ 


t Note that 12) |2 is the power spectral density of the equalizer output for the 
“rms channel.” 
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then our cost function can be rewritten in the convenient form 


n= [ WewS-[" [ Kenerwon Zs. ev 


Introducing the Hermitian integral operator X, whose kernel is K(w, v), 
permits us to write the criterion as the quadratic form 


where the inner product does not include the signal-spectrum weighting 
introduced in (10).' We now wish to minimize (22) with respect to Q, 
subject to an appropriate constraint. Referring to Fig. 1, we see that 
the average power present at the demodulator input is 


P= is [S(w)Hims(w) + N(w) ]|G(@) |? ee 


Thus, a natural constraint is to require that the power be constant. 
In terms of Q(w), this constraint takes the form 


(Q, Q) = P, (23) 


and the optimization problem consists of minimizing the positive 
definite form (22) subject to (23). The solution, which we denote by 
Qopt, is easily obtained*® by using a Lagrange multiplier and is recog- 
nized as the principal eigenfunction* of the operator K. The best filter 
shape, Gopt(w), is obtained from Qop:(w) by using (19), and can be re- 
garded as the first term in a Karhunen-Loéve representation‘ of the 
random process 
S(w) 
VS(w) Hims(w) + N(w) 


The residual value of the criterion, evaluated when Q = Qopt, 1S 
given by 


H(w). (24) 


(E)opt > (Qont, Qopt) [1 = ] = (1 —))P, (25) 


where d is the maximum eigenvalue and is a measure of how effectively 
the compromise equalizer performs for various channel ensembles and 
various signal and noise spectra. The degree to which the random 
process given by (24) is described by the first term in a Karhunen- 
Loéve expansion will, of course, determine the equalizer performance. 


t In the sequel, the inner product between the vector U and V will be taken to be 
(U, V) = SU @)V*(@) (do/2n). ee oe 

* That is, the eigenfunction corresponding to the maximum eigenvalue (it is well 
known that the eigenvalues of a Hermitian operator are real). 
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In order to get more insight into the nature of Gopt(w) we will, in the 
next paragraph, consider the form of the optimum filter under some 
special circumstances. This discussion, along with the examples treated 
in the next section, will reveal some properties of the optimum filter 
shape. 


3.2 Special Cases 


3.2.1 One Channel 


Suppose the channel ensemble consists of only one member,’ H(w). 
The principal eigenfunction of X is easily determined to be 


S(w) 
VS(w) |H(w) |? + N(@) 


Thus, the best filter shape is 


Qopt(w) = H*(w). (26) 


S(w) H*(w) 
S(@)|H(w) |? + N(@) 


The filter given by (27) is just the well-known Wiener filter for estimat- 
ing a random signal that has been passed through a nonrandom 
channel and then corrupted by additive noise. The amplitude charac- 
teristic of the filter, which is given by 


S(w)|H(@) | 
S(w) |H(@) |? + N(@)’ 


provides noise rejection at those frequencies where S(w) is small relative 
to N(w). Thus, as the spectrum of the modulating signal is changed, 
the noise-rejecting regions of the filter will be altered. It is worth 
noting that, if the channel has only phase distortion, amplitude as well 
as phase compensation is required.* It should also be noted that the 
amplitude response of the filter will be greatly attenuated at those 
frequencies where |H(w)|? dominates N(w)/S(w); this phenomenon, 
which we call channel-variance rejection, is observed in practice for an 
arbitrary channel ensemble at the frequencies where the variance of 
H(w), defined by 


Var[H(w)] = (|H(@)|*) — {H@))1?, (29) 


becomes large. Since the phase of Gopt(w), for the simple case of one 
channel, is just the negative of the channel phase, we have, under the 
further specialization of vanishingly small noise power, the not- 


Gopt(w) = (27) 


(28) 


tThis assumption gives good insight when there is small dispersion about the 
average channel, i.e., the channels all look pretty much alike. 
This, along with the preceding sentence, provides a partial answer to the second 
and third questions posed in the introduction. 
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surprising result that the equalizer should invert the channel. On the 
other hand, as the noise becomes dominant, the solution is observed to 
approach a filter matched to the corresponding average channel 
characteristic and signal spectrum. 


3.2.2 Deterministic Amplitude Distortion, Random Delay Distortion, 
and No Noise 


Suppose the members of the channel ensemble can be written as 
H(w) = a(w)e?, (30) 


where the amplitude response a(w) does not vary from channel to 
channel, and the phase response 6(w) is randomly selected. 

We consider first the kernel K( , ) when there is only delay distortion 
(i.e., a@w) = 1) and no noise. Since Hyms(w) is unity, we have 


K(a, v) — ey) (era). (31) 
and the filter is given by 
Gprtay (w) = Dovel) , (32) 


VS(w) 
If we no longer restrict a(w) to be unity, the kernel is still given by 
(31) while the filter is seen to be 


G@) = to Gpgtay(w). (33) 


The above indicates that, in the presence of deterministic amplitude 
distortion and random phase distortion, the compensation 1s decoupled 
in the sense that the filter shape is a cascade of the compensation for 
the component distortions. While this sort of decoupling is generally 
not the case, we have found in the example described in the next 
section that the phase characteristic of the equalizer is rather insensitive 
to changes in amplitude distortion and noise level. 


3.2.38 Small Variation in the Amplitude and Phase Characteristics 


We now wish to give a suggestive description of the optimum filter 
when the channel characteristics have small variation about the 
average characteristic. To this end, let us denote a typical charac- 


teristic by 
H(w) = a(w)e?™ 


E(w) b(w) e718 (+9 (0) ] (34) 


I 


I 
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Fig. 5—(a) amplitude and (b) delay characteristics for average channel. The 
wort case amplitude and phase variation are indicated by the isolated circles. 


where d(w) and 6(w) are the average channel amplitude and phase, 
respectively, and b(w) and ¢(w) represent small (random) perturbations 
about these quantities. We first write H(w) as 


H(w) = G(w)e etnd(w)+i6(w) (34a) 
= H(w)e, (34b) 
where we have let 
H(w) = d(w)e%®) (35) 
q(w) = fnb(w) + joa), (36) 


and we note that H(w) is composed of the average amplitude and phase 
of the ensemble. Since |b(w)|-41 and |¢(w)| 0, we see that 
|q(w) | = 0; thus, retaining only the leading term in the series expansion 
of e%) gives 

en) A+ glu). (37) 
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Fig. 6—Raised cosine power spectral density (rolloff = 1). 


We are now in a position to evaluate the kernel K(w, v) under the 
above assumptions. Using (37) we have 


(H*(w)H(v)) = H*(w) Hy) (eer) 
PY H*(w)H(r)(1 + g*(w) + a0), (38) 


where we have kept only first-order terms in the expansion of e*“e¢. 
If we assume that the perturbations fnb(w) and ¢(w) have zero average 
value, then (88) separates, reducing to 


(H*(w)H(v)) = H*(w)A(). (39) 

By a similar argument, the denominator of the kernel is found to be 

VS(w) |H(w) |? + N(@) X VS) |H0)|? + NO), (40) 

and combining (89) and (40) gives the separable kernel 
S(w)S(v) H*(w) H(r) 


T5041) 
VS(w) |H(w) |? + N(w)VS()|H() |? + NV) 
The equalizer shape is then given by 
Got (w) = S(w)H*(w) (42) 


S(w)|H(w) |? + No)’ 
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Fig. 7—Equalizer amplitude characteristics for various noise levels. 


which is of the same form as the one-channel result [see (27) ], as well 
as the filter obtained by Maurer and Franks (see Ref. 2). Clearly, 
for those frequencies where the noise is negligible (compared to the 
signal) the equalizer charactcristic is [1/d(w) ]Je-#™, i.e., the best 
filter is one which inverts the average channel. Recalling the first 
question posed in the introduction, we can see that, apart from noise 
rejection (which occurs in the frequency range of small signal spec- 
trum), the filter will invert the average channel when the variance of 
the ensemble is not appreciable. This is a useful rule-of-thumb for 
rapid (and approximate) compromise equalizer design. 


IV. EXAMPLES USING THE 1964 CUSTOMER LOOP SURVEY 


In this section we consider the design of a compromise equalizer for 
use over an ensemble of data customer loops. Some knowledge of the 
characteristics of this ensemble can be obtained from the 1964 Loop 
Survey.> This survey collected information about transmission param- 
eters and channel makeup (e.g., gauges and lengths of sections which 
compromise the loop, locations of load coils and bridged taps, etc.) for 
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Fig. 8—Average channel delay (inverted) and equalizer delay (SNR = 30 dB). 


a random selection of loops. For our purposes we extracted an ensemble 
of business loops (as opposed to residential loops) described in the 
survey. An existing computer program was used to obtain the fre- 
quency responses of the channels (with load coils and bridged taps 
removed) from the descriptions of the physical makeup of the channels. 
A recent study® shows that, at voiceband frequencies, the functions 
{e-Viee} 9 provide a good approximation of the loop transfer charac- 
teristics. The parameter a is a random variable (whose distribution 
can be approximated by the survey information), specifically a = RC¢?, 
where F# is the average series resistance per mile, C is the capacitance 
per mile, and ¢ is the loop length in miles. The average channel is 
depicted in Figs. 5a and 5b in terms of gain and delay.t These loop 
characteristics display appreciable amplitude variation (as well as. 
delay variation) and the extent of these variations is indicated by the 
isolated circles. 

To illustrate the design technique on these loop networks, we deter- 
mined a compromise equalizer characteristic to be employed in a 
maxentropic 4.8-kb/s stream of randomly signed pulses. The basic 


t The channel delays each include an estimate of Bopt. 
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Fig. 9—Reduction of residual error as a result of equalization. 


pulse was assumed to have a raised cosine spectrum, and the resulting 
power spectral density is shown in Fig. 6. It was assumed that only 
the “‘worst’’ 70 of the 143 customer loops required equalization. Figure 
7 shows the resulting equalizer amplitudes for various noise levels. In 
this example, the channels were normalized to avoid the ‘‘whomper’”’ 
effect mentioned in Section II. So when the signal-to-noise ratio is not 
infinite, the noise levels should be interpreted as if the lowest signal- 
to-noise ratio for the entire ensemble was assumed for all channels. 
Observe that, in the absence of noise, the equalizer tends to invert the av- 
erage channel amplitude characteristic, and as the noise level is increased 
the equalizer tends to attenuate the high end of the spectrum where the 
signal power is lowest. Also note that, as the signal-to-noise ratio de- 
creases, the solution tends to a filter matched to the raised cosine 
characteristic times a characteristic approximating the average 
channel. We observed that, for each of the noise levels, the compromise 
equalizer delay is close to the inverse delay of the average channel. 
The specific delay curve for the case of 30-dB SNR is provided in Fig. 8, 
which also shows the average channel delay (inverted). Finally, a 
direct computation shows that, for typical signal-to-noise ratios, the 
equalizer reduces the residual error by 4.8 dB, and Fig. 9 displays this 
improvement in mean-square error as a function of signal-to-noise 
ratio. The improvement is greatest at high SNR since the equalizer is 
combating only linear distortion rather than a combination of linear 
distortion and noise. 
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VI. CONCLUSIONS 


Using the second-order statistics of the channel ensemble, as well as 


the signal and noise spectra, the equalizer characteristics were easily 
computed by solving a matrix eigenvalue problem. Several interesting 
conclusions can be drawn concerning the properties of the equalizer: 


(t) The equalizer amplitude is attenuated in those frequency 
regions where the signal-to-noise ratio or signal-to-channel- 
variance ratio is small. 

(it) When the channel ensemble has only delay distortion, ampli- 
tude as well as phase compensation is required. 

(477) The delay characteristics of the equalizer are rather insensitive 
to changes in noise level or (non-random) amplitude distortion. 

(tv) When the channel ensemble has small variation about the 
average characteristic, a situation that commonly arises in 
practice, then the equalizer will invert the average channel. 
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Distribution of La, /n, a, Randomly 
Equal to +1 


By S. O. RICE 
(Manuscript received February 6, 1973) 


When ai, 2, ++: are independent random variables, each equal to +1 
with probability 4, the sum >? a,/n is a random variable whose distribu- 
tion is dtifficult to determine theoretically. This sum is of interest in the 
study of intersymbol interference in digital communication systems. Here 
the distribution of the sum is computed by numerical integration and the 
results tabulated. Asymptotic expressions are given for the tails of the 
distribution. 


I. INTRODUCTION 


The distribution of the random variable 


= ¥ a,/n, (1) 


where 41, a2, --- are independent random variables equal to +1 or 
—1 with probability 3, is of some interest in the study of intersymbol 
interference in a digital communication system. For example, the sum 
of two independent expressions of the form (1) occurs when the pulse 
train >°%,. ad, sin (t — nr)/(t — nm), Gn randomly equal to +1, is 
sampled at regularly spaced instants which are slightly out of step with 
the zeros of sin ¢. The theory of random variables of type (1) (in par- 
ticular with a,@” in place of a,/n) has been studied by a number of 
investigators. A survey of the field has been made recently by Hill and 
Blanco.! Here we evaluate the distribution of x numerically and give 
expressions for its behavior when 2 is large. Questions of continuity 
and convergence are put aside. 

Since the distribution is even about x = 0, only values for x = 0 
need be considered. From the characteristic function 


f(u) = avg Lexp (éeu)] = II Leos (u/n)] (2) 
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we get an expression for the probability density p(x) of x: 


pa) = = [cos (eu) fludeu, (3) 
Prob (@ > #1) = ; — A pase f(u)du. (4) 


The values of p(a) and Prob (a > 21) shown in Table I were obtained 
by evaluating these integrals by the trapezoidal rule?? which works well 
for (8) and (4). 

The asymptotic expressions (8) and (18) for p(x) follow from a 
saddle point analysis of (3). Both p(x) and Prob (x > 21) decrease 
rapidly when x (or x1) > 3, the decrease being dominated by the factor 
exp [—exp(« — A) ] where A = 1.39---. 

The rapid decrease of p(x) is interesting because the divergence of 
> 1/n might lead one to expect that p(x) would decrease slowly as 
x—c, Instead, p(x) actually decreases much faster than a Gaussian 
probability density. I am indebted to a referee for the observation 
that the second, fourth, and sixth moments of p(x) are, respectively, 
m?/6, 1144/180, and 23375/7560. . 

The reader may wonder why as many as six decimal places are given 
in Table I. There are several reasons. One is that the cost was low. 
About 3 seconds were required by a Honeywell 6000 Processor to 
compute the values shown in Table I, and about 40 terms were required 
in each trapezoidal sum. This illustrates the fact (apparently not well 
known) that when integrals like (3) and (4) are to be evaluated 
numerically, the trapezoidal rule often performs better than most of 
the other conventional quadrature methods (better than Simpson’s 
rule, for instance).? The six-figure accuracy is also used to gain an idea 
of the values of « for which the asymptotic expansion (18) for p(x) 
begins to be valid. This degree of accuracy also shows that p(0) is 
equal to 0.249 994 --- and not to 4, as might be inferred from a four- 
figure tabulation. 


II. TRAPEZOIDAL RULE CALCULATION 


Preliminary computations showed that | f(w)| < 107-!°whenw > 15. 
Furthermore, it was found that (3) and (4) could be evaluated to 
within the desired accuracy by using a trapezoidal-rule spacing of 
Au = h = 0.4. In line with these values the trapezoidal sum was 
truncated at the 40th term (15/0.4 = 40). 
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TABLE I—VALUES OF p(x) AND Prob (x > 21) 





x or 21 p(x) Prob (x > 21) zor 21 p(x) Prob (2 > 21) 
0.0 0.249 994 0.500 000 2.0 0.125 000 0.056 599 
0.2 0.249 970 0.450 003 2.2 0.091 768 0.034 949 
0.4 0.249 802 0.400 021 2.4 0.061 647 0.019 683 
0.6 0.249 073 0.350 118 2.6 0.037 148 0.009 912 
0.8 0.246 778 0.300 494 2.8 0.019 592 0.004 357 
1.0 0.241 222 0.251 623 3.0 0.008 777 0.001 623 
1.2 0.230 408 0.204 357 3.2 0.003 222 0.000 494 
1.4 0.212 852 0.159 912 3.4 0.000 927 0.000 118 
1.6 0.188 353 0.119 683 3.6 0.000 198 0.000 021 
1.8 0.158 232 0.084 949 3.8 0.000 029 0.000 003 
2.0 0.125 000 0.056 599 4.0 0.000 003 0.000 000 








The infinite product (2) for f(u) was computed by using 


f(u) = (11, [cos (u/n)]) exp ee In cos wn), (5) 


where JN is a large number such that u/N <1 for all values of u used 
in the computation (0 S$ u S 16). The product [J?’~’ in (5) was 
computed by straightforward multiplication. The sum in (5) was 
computed by setting g(n) = In [cos (u/n)] in the Euler-Maclaurin 
sum formula: 


& a(n) = f° ae + 50(N) — Fg) 


_B Box 


(2k) | 





m9 = oot g?-)(N) + Re. (6) 
Here the B’s denote Bernoulli’s numbers, Bz, = 1/6, Bs = — 1/30, 
= 1/42 [we stopped at Bs in our use of (6) ], g“(N) denotes the 
value of (d/dt)*g(t) at ¢ = N, and the remainder R, is the integral of 
g?*+(¢) times the Bernoulli “polynomial” of degree 2k +1 and 
period 1 (see pages 520-540 of Ref. 4). 
The integral in (6) can be evaluated to within the desired accuracy 
by setting u/t = y, dt = — udy/y?, expanding In (cos y) in powers of 
y with the help of 


17v’ 


—In (cosy) = [tan odo = [” jo+ 3 +340 


+: | a, 
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and integrating termwise: 


oo) ulN 
J g(t)dt = u | y~* In (cos y)dy 
N 0 
u? ut us 17u8 
= on * 36Na 1 295Ne * 17640N7 


Expressions for the higher derivatives of g(V) in (6) can be obtained by 
differentiating the series in (7) with respect to N. 

In using (6) we stopped at k = 3 and neglected R3. 

Three separate trapezoidal-rule evaluations of p(x) and Prob (x > 21) 
were made using eqs. (8) to (7) with h = 0.4, N = 201; h = 0.38, 
N = 201; and h = 0.36, N = 301, respectively. Here h is the spacing 
used in the trapezoidal-rule evaluations of (3) and (4). The three sets 
of computed values differed only in the 7th or 8th decimal places, 1.e., 
all agreed with the values shown in the table. The values 201 and 301 
of N are so large that terms beyond k = 3 in (6) and those shown in 
(7) are not needed. To check the computations, the integrals of p(z) 
amd 2’p(x) from x = 0 to x = ~ (the upper limit used was actually 
x = 5) were computed by the trapezoidal rule with a spacing of 
Az = 0.1. The trapezoidal values agreed with the known values, 
respectively 4 and 7?/12, to within 6 significant figures or better. 








pete, OD 


III. DISCUSSION OF TABLE I 


Table I shows that p(x) remains nearly equal to p(0) = 0.249994 for 
0 S x S 1, passes through p(2) = 0.125000 (is it exactly #?), and then 
decreases rapidly. The question as to whether p(2) is exactly $ remains 
unanswered, but p(0) = 0.249994 does not seem to be an erroneous 
calculation of 4. For if p2(u) is the probability density of u = )°9 an/n, 
then 


p(x) = 2p2(e — 1) + gpo(w + 1). 


Setting « equal to 0 and 2 and combining the results give a result I 
owe to J. E. Mazo, 


p(2) = 3p(0) + 3p2(8) > 3p(0). 
Furthermore, replacing $p2(8) by p(4) — 4p2(5) gives 
p(2) = zp(0) + p(4) — zp2(5) 
0.125000 = 0.124997 + 0.000003 — }$p2(5) 


which is satisfied by the tabulated values when p2(5) is assumed to 
be negligibly small. 
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IV. ASYMPTOTIC EXPRESSIONS FOR LARGE 2 

We shall show that the rapid decrease of p(x) for 2 > 3 is described 

by 

p(x) ~ (Yyo/m)te~%, 

yo = exp [x — 2y + In (7/4) ] = exp [a — 1.39599 -- -], 
where y denotes Euler’s constant, 0.577215 ---. Integrating (8) gives 
Prob (2 > a1) ~ erfe (y3) ~ p(21)/Yyo, (9) 
where yo is computed from the second of equations (8) with 2; in place 
of 2. Bounds for the distribution involving exponential functions of 


e* have been obtained by L. A. Shepp in unpublished work. 
To obtain (8) we rewrite the integral (3) for p(2) as 


p(a) = 5- fe fwdu, (10) 


As is often the case for such integrals, the asymptotic value of p(x) is 
given by the contribution of a saddle point, wo = zy, lying far out on 
the positive imaginary u-axis (the path of integration being deformed 
so as to pass through the saddle point). For u = zy the integrand in 
(10) becomes 


expC—ay + oy], oY) = LX In[eosh (y/n)]. (A) 


When y is large we can show that 


ey) =ylny—ytAyt7ln2 + ry), 
A = 2y — In (7/4) = 1.89599 ---, 

where r(y) has roughly the same magnitude as exp (—2y). 

We first outline the derivation of the expression (12) for ¢(y), and 
then apply (12) to obtain the asymptotic expression (8) for p(z). 

The derivation of (12) is based upon the Euler-Maclaurin sum 
formula (6) with N = 1 and g(t) = In [cosh (y/t) ]. The integral in the 
sum formula is 


im In [cosh (y/t) ]dt 


(8) 


(12) 


y in v-? In (cosh v)dv 
0 


I 


— In (cosh y) + y(In y) tanh y — y in (In v) sech? vdv 
0 


II 


n2—yt+ylny—y is (In v) sech? vdv + O[ye-*¥ In y], (13) 
ct) ; 
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where we have integrated by parts twice. The last integral in (13) has 
the value 


i " (nv) sech? ody = — y + In (x/4) (14) 


which can be obtained by (2) replacing In v in (14) by v#7!, (22) differ- 
entiating the known value (formula 3.527-3, page 352 of Ref. 5) of the 
resulting integral with respect to yw, and (wz) setting » = 1. The 
derivative of g(t) with respect to ¢, g(@), in the sum formula (6) is 


g(t) =— yt tanh (y/t) 
ee yt(1 — Qe-2u/t + De-sylt —.. *); 


where the exponential terms become negligible when y becomes large 
and ¢ = 1. In general, for 1 = 0, 1, 2, ---, g@4(1) is equal to 
— (21 + 1)! y plus negligible terms. Therefore, the right side of the sum 
formula (6) is the integral plus 


My — In2) + Sy + Sty bet oR (15) 


plus terms which are negligible when y is large. The sum of the co- 
efficients of y in (15) is known to be equal to y (page 529 of Ref. 4). 
Hence (15) is equal to yy — $1n 2. Addition of (18) and (15) and use 
of (14) gives the expression (12) for ¢(y). 

Next we use the expression for ¢g(y), with the small term r(y) 
neglected, to obtain the asymptotic form of p(x). The saddle point of 
interest occurs at Uo = 7y¥o Where yo is the zero of the derivative of the 
exponent in (11). The exponent is —zy + ¢(y), and yo is the zero of 


—a2+e'(y)=—-ax2+Inyt+A. 


Thus yo = exp ( — A). This yo is the same as the yo appearing in the 
asymptotic expression for p(x) stated in (8). The exponent itself has 
the value —xyo + (yo) =— yot$ln2 at yo. By making use of 
ge’ (y) = 1/y and the higher derivatives of ¢(y), the exponent can be 
expanded in a Taylor series about yo. From this expansion it follows 
that near uo the integrand in the integral (10) for p(x) can be written as 





_ 1 = _(72/Yo)* 

exp | yo to in2 + yore = |: (16) 
where z = u — Uo. Setting (16) in (10), changing the variable of 
integration from u to 7 = z2/yo = (u — Uo)/Yo, and assuming that p(x) 
is given asymptotically (as x and yy tend to ~) by the contribution of 
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the saddle point at uo give 


p(x) ~ (Qm)-!2kype-™ / exp | vo eon | ae (17) 


R= 
Here the nominal path of integration is the real r-axis. The classical 
saddle point asymptotic expansion obtained from (17) is 


1 23 |. 


p(z) ~ (yo/m) te~¥? E + My, — Ti52y8 2 ee (18) 


0 

The coefficients of the powers of 1/yo in the series can be determined by 
a general procedure described in Appendix D, page 1999, of Ref. 6. 

The asymptotic expression stated in (8) is the leading term in (18). 
An idea of the accuracy of the asymptotic expressions can be obtained 
by considering the case x = 3. For x = 3, yo is 4.973 and Table I gives 
the ‘exact’? value p(3) = 0.008777. The asymptotic value of p(3) 
obtained from (8) is 0.008710, the first two terms in (18) give 0.008783, 
and the first three terms give 0.008776. Table I gives the ‘“‘exact’’ 
value Prob (2 > 3) = 0.001623 and eq. (9), namely Prob (x > 3) 
~erfe (yj) = erfe (2.230), gives 0.001612. 
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Adaptive Quantization in Differential 


PCM Coding of Speech 


By P. CUMMISKEY, N. 8. JAYANT, and J. L. FLANAGAN 
(Manuscript received March 12, 1973) 


We describe an adaptive differential PCM (ADPCM) coder which 
makes instantaneous exponential changes of quantizer step-size. The coder 
includes a simple first-order predictor and a time-invariant, minimally 
complex adaptation strategy. Step-size multipliers depend only on the 
most recent quantizer output, and input signals of unknown variance can 
be accommodated. We derive appropriate multiplier values from com- 
puter simulations with speech signals and with Gauss-Markov inputs. We 
compare performance of the ADPCM coder with conventional log-PCM, 
using both objective and subjective criteria. Finally, we describe an 
economical integrated hardware implementation of the ADPCM coder. We 
believe that at bit rates of 24 to 82 kb/s, ADPCM provides a robust and 
efficient technique for speech communication and for digital storage of 
speech. 


I. INTRODUCTION 


The advantages of coding speech digitally are well known.! Expected 
benefits include low costs per line, ease of maintenance, and high- 
quality signal regeneration at repeaters. Furthermore, digital coding is 
well matched to current technology in terms of readily available 
integrated circuit hardware. Results from speech-coding research are 
now beginning to specify techniques that are nearly optimal for a given 
bit rate, a given channel quality, and a given degree of coder complexity. 
Finally, the subject of direct digital conversion between alternative 
code formats is being widely studied, and simple techniques have 
already been proposed for some specific conversions. 

The coder discussed in this paper is believed to be efficient and 
robust for speech coding at bit rates of 24 to 32 kilobits/second (kb/s). 
Other refinements of differential PCM (DPCM)? are based, at least in 
part, on adaptive prediction.*~® These techniques offer considerable 
potential for bandwidth compression,’ but are typically hard to 
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implement. Therefore, for the type of bit rates mentioned earlier, it 
seems much more reasonable to tap the advantages of a more simply 
implemented adaptive quantizer. 

Our adaptive DPCM (ADPCM) coder, therefore, operates on the 
basis of a fixed, first-order predictor in the DPCM loop, and a time- 
invariant, adaptation strategy for instantaneous changes of quantizer 
step-size. The technique has obvious advantages over conventional 
PCM (due to redundancy removal) and over conventional DPCM 
(due to increased dynamic range). Further, the quality of speech 
reproduction in the 24- to 32-kb/s range is believed to be perceptually 
better than that provided by adaptive delta modulation (ADM) which, 
however, has the advantage of even greater simplicity.” 

Besides digital telephone applications, appropriate utilizations of 
ADPCM coding are seen in computer storage of digital speech (for voice 
answer-back, ‘‘voice-wiring,” and similar functions), in mobile radio 
telephony, and in special applications such as deep-space communica- 
tion and digital encryption. 


II. DEFINITION OF THE ADPCM CODER: ADAPTIVE QUANTIZATION WITH A 
ONE-WORD MEMORY 


A schematic block diagram of the coder appears in Fig. 1. It follows 
the conventional differential PCM structure with a first-order, fixed 
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Fig. 1—Block diagram of ADPCM coder. 
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predictor in the feedback loop.? It has, however, the additional box 
labeled LOGIC, which provides adaptation of quantizer step-size on 
the basis of the most recent quantizer output. In the absence of channel 
errors, the step-size controls « and o’ are identical, and so are the 
signal estimates ¢ and 2’. 

Step-size adaptations are motivated by the assumption that the 
variance of the quantizer input 6 is unknown. The empirical adaptation 
rule is that, for every new input sample, the step-size is changed by a 
factor depending only on the knowledge of which quantizer slot was 
occupied by the previous signal sample. 

Formally, if the outputs of a uniform B-bit quantizer are of the form 


pi 06 Py H1,3,---2?-1; Au>0, 


the step-size A, is given by the previous step-size multiplied by a time- 
invariant function of the code-word magnitude |P,-1| ; 


As = A,-1:-M(|P,-1|), 


subject, of course, to maximum and minimum limits on A,, as specified 
in specific implementations. Step-size multipliers for a 3-bit uniform 
quantizer are illustrated in Fig. 2. Note that there are only four distinct 
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Fig. 2—Adaptation multipliers associated with quantizer levels for 3-bit ADPCM 
coder. 
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TABLE J—ADAPTATION MULTIPLIERS FOR SPEECH AND FOR GAUSS- 
Markov Inputs (3-Bit CopER) 


Gauss-Markov Input 
(Correlation Between 
Previous Output Word Multiplier | Speech Input | Adjacent Samples = 0.5) 


111 or 000 Ms 2 1.75 
110 or 001 M; 5/4 1.25 
101 or 010 M2 7/8 0.90 
100 or 011 Mi 7/8 0.90 


multipliers because the polarity of quantizer output is not utilized in the 
adaptation logic. Furthermore, meaningful adaptation requires that 
the step-size be increased on the detection of quantizer overload 
(M,> 1) and decreased during underload (Mi <1), and that 
M, Ss Mz. S M3; S My. Derivation of specific multiplier values is out- 
lined in the next section. 


III. DESIGN OF STEP-SIZE MULTIPLIERS 


Two conflicting requirements are encountered in designing step-size 
multipliers. The first is the need to respond quickly to abrupt changes 
of input variance (suggesting the use of Ms, >>1, M1 < 1 for the 3-bit 
example). The second requirement is the prevention of excessive 
step-size alterations in a stationary or steady-state situation (suggest- 
ing the use of M,=1 +4; My; = 1— 1; 4, , ~0). Compromise 
values of multipliers are therefore suggested for an input signal, or for 
a class of input signals. 

Extensive computer simulations were carried out to determine the 
most desirable multiplier values for an illustrative speech sample. The 
sample was a male utterance of ‘‘This circuit operates on the same 
principle as N. 8. Jayant’s simulation.”’ The speech was bandpass- 
filtered (200-3200 Hz), and was sampled at 8 kHz. Multiplier values 
were sought that maximized the signal-to-quantization-error (power) 
ratio (SNR), as averaged over the entire duration of the above utter- 
ance. Rounded values of these multipliers are shown in Table I for a 
3-bit quantizer. Also shown are the values found to be desirable for the 
quantization of a Gauss-Markov input with an input signal correlation 
similar to that expected for Nyquist-sampled speech.? The similarity is 
interesting, particularly because the speech quantizer had a Max 
nonuniformity® (to take into account the observed Gaussian tendencies 
of the quantizer input), while the Gauss-Markov simulation utilized a 
uniform quantizer. The latter simulation also showed that desirable 
multiplier values are only slightly dependent on signal correlation, 
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Lev ADPCM 8-kHz SAMPLING 
LOG-—PCM 8-kHz SAMPLING, 
Om—=O » = 100 


SIGNAL — TO — QUANTIZING — NOISE RATIO IN DECIBELS 
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BIT RATE IN kb/s 


Fig. 3—Signal-to-quantizing-noise ratios for speech signals. 


suggesting a robust adaptation strategy. Finally, the similarity of the 
multiplier values found for speech and for Gauss-Markov inputs 
suggests that the coder has a versatility that might extend to facsimile 
and video signals.?° 

The general problem of determining most desirable multiplier values 
is discussed at length in a companion paper.!® That paper also points 
out the possibility of near-optimal adaptation strategies that have 
nontrivial (+1) values only for the end multipliers (4, and M, in the 
3-bit example), and compares our adaptation logic with that of Stroh.’ 


IV. PERFORMANCE COMPARISONS OF THE ADPCM CODER WITH CON- 
VENTIONAL PCM 

4.1 SNR Data 

Computer simulations using speech input showed the signal-to-error- 
power ratio to be 16 dB for a 3-bit (24 kb/s) ADPCM coder which 
has the multipliers of Table I and a maximum step-size which was 
D = 128 times the minimum. The SNR of 16 dB represents an 8-dB 
gain over 38-bit logarithmic PCM with uw = 100. It turns out that 


* This value was chosen on the basis of past experience with speech coders. Present 
trends are toward a higher value for uz. 


1110 THE BELL SYSTEM TECHNICAL JOURNAL, SEPTEMBER 1973 


TaBLE II—CoMPARISON OF OBJECTIVE AND SUBJECTIVE 


PERFORMANCE OF ADPCM anp Loc-PCM 
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FREQUENCY IN kHz 


Objective Rating Subjective Rating 
(SNR) (Preference) 
7-bit PCM 7-bit PCM (High) 
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4-bit ADPCM 6-bit PCM 

Sbit PCM =e Sit ADPCM 

3-bit ADPCM 5-bit PCM 

4-bit PCM 4-bit PCM — (Low) 
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Fig. 4—Spectograms of speech and quantization error. 
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this improvement includes a 4-dB gain due to differential encoding and 
a 4-dB gain due to the quantizer adaptation. Figure 3 gives a more 
complete comparison of speech signal SNR’s measured for the ADPCM 
and for log-PCM. 


10 
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MAX = 11.342035 
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ADPCM QUANTIZING 
NOISE POWER SPECTRUM 
MIN = —2.470775 

MAX = 2.020871 


10 LOG, (e2) IN dB 





POWER SPECTRUM LOG— 
PCM ERROR SIGNAL 
MIN = —1.457203 

MAX = 3.259536 


10 LOG, 9.(e2!) IN dB 





FREQUENCY IN kHz 


Fig. 5—Long-term error spectra. 
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4.2 Error Spectrograms and Long-Term Spectra 


Figure 4 displays spectrograms of a section of the input speech and 
the associated quantizing noise spectrograms with 3-bit ADPCM and 
5-bit log-PCM. Note that ADPCM provides considerably less noise 
during the silent intervals in speech, although the total noise power is 
greater in this coder (Fig. 3). This suggests that adaptive quantization 
in ADPCM provides greater dynamic range than the logarithmic com- 
pandor used in PCM. (The observation results, no doubt, from the 
specific numerical values » = 100 and D = 128 in Section 4.1. How- 
ever, these values are believed to be very representative.) 

Figure 5 illustrates another interesting difference between the 
quantizing noise in ADPCM and that in PCM. Note the high-fre- 
quency rolloff in ADPCM noise and the relative whiteness of the noise 
spectrum in PCM. 


DIM 2 
DIM 3 OUT OF PAPER 
®6 PCM 
Dé 
& 
é 
S @5 PCM 
3 0 5 
~“t ~~), appan— TSS 3 ADPCM 
-1 DIM | 


@7 ecm 


4 PCM 


Fig. 6—Subjective preference judgments of various ADPCM and log-PCM 
codings. Dimensions 1 and 2 account for most of intersubject variance. Increasing 
preference is in the —z direction. Individual subject vectors are plotted, and projec- 
tion of coding conditions onto a subject vector indicates how that individual rank- 
ordered the coding systems. 
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4-—BIT ADPCM 


PERCENTAGE OF TIMES ADPCM IS PREFERRED 





LOG-—PCM BITS 


Fig. 7—Alternative interpretation of (ADPCM-versus-PCM) preference scores. 


4.3 Subjective Tests 

Apart from the measured differences mentioned in Section 4.2, 
informal listening tests indicated that ADPCM noise had a per- 
ceptually more palatable character* than PCM noise of equal variance; 
in other words, the quality of speech reproduction from the ADPCM 
(relative to PCM) was much better than was suggested by the SNR 
comparisons in Fig. 3. This observation was borne out in the following 
perceptual experiment. 

The experiment involved 3- and 4-bit ADPCM stimuli and 4-, 5-, 
6-, and 7-bit log-PCM stimuli. The total number of cross comparisons 
possible was 16 (2 stimuli X 4 stimuli X 2 orders of presentation). 
Twenty-two listeners participated in the tests and made preference 
judgments of signal quality for each of the 16 A-B comparisons. The 
preference Judgments were submitted to a multidimensional scaling pro- 
gram,” and the results were plotted in terms of two subjective dimen- 
sions which accounted for most of the perceived differences. Dimension 
1, in particular, accounted for 75 percent of the variance in the prefer- 
ence data. 

The results are shown in Fig. 6. Individual subject vectors are 


* Related, perhaps, to a lesser proportion of the noise getting into the idle circuit ; 
and, also, due to some correlations of ADPCM noise with pitch information. 
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Fig. 8—Circuit block diagram for the hardware ADPCM coder. 


displayed (solid lines), and projection of the coding conditions into a 
subject’s vector reveals how that individual rank-ordered the signal 
qualities.* A resultant of the subject vectors is also shown (dashed), 
and projections onto this resultant indicate subject consensus in rank- 


_* One subject (vector in quadrant IV) apparently misunderstood the test instruc- 
tions and gave essentially complementary preference judgments. 
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Fig. 9—ADPCM coder. 


ing the qualities. This averaged subjective ranking can be compared 
with the objective SNR performance for the same codings (using the 
data of Fig. 3). 

A comparison of the objective (SNR) performance and the sub- 
jective (preference) ranking of the codings tested is shown in Table II. 
It is clear from these data that subjectively the ADPCM does an even 
better job than the objective SNR’s indicate. Arrows indicate two 
subjective ‘‘promotions” of the ADPCM. Table II (and, of course, 
Fig. 6) show, too, that 4-bit ADPCM is perceptually better than 
6-bit log-PCM. 

Figure 7 provides yet another means of comparing ADPCM and 
PCM using the original preference scores from the perceptual test. 
Ordinates in Fig. 7 are overall percentages (including all listeners) of 
A-B judgments where an ADPCM stimulus was preferred to a certain 
PCM stimulus as shown on the x-axis of Fig. 7. The 50-percent prob- 
ability-of-preference line intersects the curves at points whose ab- 
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SINE-WAVE SIGNAL — TO — NOISE RATIOS 
HARDWARE ADPCM CODER 

4 BITS/WORD, 16-LEVEL LINEAR QUANTIZER 
SAMPLING RATE 6 kHz, BANDWITH 200 Hz — 2.8 kHz 


SNR IN DECIBELS 





SIGNAL LEVEL IN dBm 


Fig. 10—Signal-to-quantizing-noise ratios for sine-wave input to the hardware 
ADPCM coder. 


scissas represent quantitative log-PCM equivalences for 3- and 4-bit 
ADPCM. 


Vv. HARDWARE DESIGN OF ADPCM CODER 


The computer simulations described above established the design 
criteria for the ADPCM. To assess hardward viability of the technique, 
we constructed a 4-bit ADPCM coder in integrated circuit hardware. 
A circuit block diagram of the hardware coder is given in Fig. 8, and a 
photograph of the circuit card is shown in Fig. 9. State-of-the-art 
circuit technology is used and all circuit components are ‘‘off the shelf.” 

The circuit incorporates a uniform quantizer realized by using a serial 
logic (shown at the bottom of Fig. 8) to code the difference between 
the input X and the signal stored on the integrator Y. The logic 
provides four consecutive increments of integrator voltage within a 
duration much smaller than the sampling period. Each of these in- 
crements follows the latest sign of (X — Y) and has a magnitude that 
is one-half that of the previous increment in the cycle. 

Step-size adaptations are controlled by the current switches which 
provide a dictionary of 21 step-sizes. These are spaced with a ratio of 
2! between adjacent steps, and therefore provide an overall step-size 
range of 128:1. The step-size multipliers for speech (Table I) are 
approximated in the circuit as positive and negative exponents of 23. 

Measurements on the hardware realization confirm the computer 
observations on SNR for speech signals. Also, SNR’s for sine-wave 
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inputs are conventionally used to assess digital coders (although sine- 
wave performance can be very deceptive in terms of perceptual 
acceptability). Figure 10 shows SNR measured on the hardware coder 
for sine-wave input. The 800-Hz behavior is reasonably consistent with 
the SNR measured for speech input. 


VI. DIRECT DIGITAL CONVERSION BETWEEN ADPCM AND OTHER SIGNAL 
FORMATS 


An important issue in the compatibility of digital systems is the 
provision of graceful and virtually transparent conversions among 
different code formats. Digital techniques for directly converting 
between ADPCM and the conventional formats of DPCM and PCM 
have been proposed and are being studied.“ One of the indications of 
these studies is that direct conversion between ADPCM and DPCM 
is quite feasible, especially when the conversion incorporates an 
intermediate stage of PCM. 


VII. CONCLUSION 


Results of this study indicate that the ADPCM technique leads to an 
economic, efficient digital coding of speech for the bit-rate range 24 to 
32 kb/s. This range constitutes a channel capacity saving of over 2:1 
compared to conventional PCM and produces a signal coding of com- 
parable quality. Hardware implementation is relatively straightforward 
and noncritical. 

Studies presently in progress are examining ADPCM coding for 
operation at 18 kb/s. Preliminary indications are that signal quality 
attractive for mobile radio application can be achieved at this low bit 
rate. This low rate also makes digital encryption for privacy attractive 
in mobile telephone. 

Although not specifically discussed in this exposition, ADPCM 
proves reasonably robust in the presence of errors in the transmit 
channel. The computer simulations described above incorporated pre- 
liminary studies of error vulnerability which show the coder to perform 
well for channels with error probability <10-4. Typical error rates in 
“clean”? PCM channels are routinely maintained lower than this. 

Further study of ADPCM is anticipated in objective analysis of its 
quantizing characteristics. This should be coupled with more complete 
perceptual tests to better understand the “perceptual palatability” of 
ADPCM coding. Further, a close competitor is adaptive delta modula- 
tion,’:® and subjective comparisons are planned that will include this 
coding technique. 
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One present utilization of the hardware ADPCM coder is in a 
computer voice response system for generating computer-spoken wiring 
instructions.15 Speech coding at 24 kb/s provides economy of digital 
storage and simple A/D—D/A communication with the computer. 
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Adaptive Quantization With a One-Word 
Memory 


By N. S. JAYANT 
(Manuscript received March 12, 1973) 


We discuss a quantizer which, for every new input sample, adapts its 
step-size by a factor depending only on the knowledge of which quantizer 
slot was occupied by the previous signal sample.! Specifically, of the out- 
puts of a uniform B-bit quantizer (B > 1) are of the form 


Au 


Yu= Pus 


; +P, = 1,3,--: ,2? —1; A, > 0, 
the step-size A, is given by the previous step-size multiplied by a time- 
invariant function of the code-word magnitude |P,_1| : 


Ap Aegs (PLAN. 


The adaptations are motivated by the assumption that the input signal 
variance ts unknown, so that the quantizer is started off, in general, with a 
suboptimal step-size Astarr. Multiplier functions that maximize the 
signal-to-quantization-error ratio (SNR) depend, in general, on Agrarr 
and the input sequence length N. For example, if the signal is stationary 
and N —», best multipliers, irrespective of Asrart, have values arbi- 
trarily close to unity. On the other hand, small values of N and suboptimal 
values of Asrarr necessitate M values further away from unity. By 
including an adequate range of values for N and Asrarrt tn a generalized 
SNR definition, we show how one can determine stable multiplier functions 
Morr that are optimal for a given signal. 

In computer simulations of 2- and 38-bit quantizers with first-order 
Gauss-Markovian inputs, we note that, except when the magnitude of the 
correlation C between adjacent samples is very high, Mopr has the property 
of calling for fast increases and slow decreases of step-size. We derive 
optimum multipliers theoretically for two simple cases: 
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K is a constant depending only on B, and & is a positive correction that ts 
significant only for the last slot: |P,1| = 28 — 1. Using the example of 
C = 0, we also show how the approach of specifying P,-1, explicitly, in 
the determination of A,, 1s more effective than an earlier procedure? where 
A, is determined by past output values Y,-1 (rather than by a function of 
their components, P,1 and A,_1). 

Computer simulations with speech and picture signals have shown, once 
again, that SNR-maximizing multiplier functions demand step-size in- 
creases that are relatively faster than step-size decreases. Values of Mopr 
depend, interestingly, on whether the quantizer is used in a PCM or a 
DPCM-type coder. In the case of speech signals, we propose corresponding 
tables of Morr values for B = 2, 8, 4, and 5. DPCM coding of speech 
with 8- and 4-bit adaptive quantizers is the subject of a companion paper. 


I. INTRODUCTION 


Quantization error, in general, can take one of two distinct forms, 
overload distortion or granular noise, reflecting, respectively, situations 
where the quantizer step-size is too small or too large relative to the 
signal being quantized. This distinction has been widely noted for 
1-bit quantizers (delta modulators), and variable step-size quantization 
has therefore been widely discussed in this context.3-® The general idea 
is to increase the step-size during overload and decrease it during granu- 
larity, and to detect those conditions on the basis of observations of the 
delta modulator bit stream. The step-size adaptations can be either 
instantaneous*'®:* or “‘syllabic,’’* and the advantages of adaptation have 
been shown, among other means, by demonstrations of dynamic range 
and of SNR gains over nonadaptive quantizers.°® 

The problem of step-size adaptations, as applied to quantizers with 
more that two output levels, has been less widely studied. It is con- 
ventional in such quantizers to take signal nonstationarity into account 
by means of a suitably designed, time-invariant, nonuniform quan- 
tizer.’ Recently, however, two proposals have incorporated time- 
variant step-size logics in multibit quantization. The first of these 
techniques is a syllabically adapting PCM which Wilkinson empirically 
designed for speech encoding at 10 kb/s.§ The second proposal is an 
instantaneously adapting quantizer discussed by Stroh, in the context 
of differential encoding of Gaussian signals.? Syllabic adaption has the 
advantages that it can be better tailored to a given signal such as 
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speech and that it can also be designed to provide better resistance to 
bit errors‘ than instantaneous adaptation. The latter, on the other 
hand, has the advantages of minimal structure and applicability to 
different types of signals, and, in relatively noise-protected environ- 
ments, it constitutes an efficient and simple encoding procedure for 
signal storage or transmission. 

The adaptation that we discuss is instantaneous, and we indicate, 
at the end of this paper, how it can perform better than Stroh’s com- 
pandor? when working with one word of quantizer (output) memory. 
We must emphasize here that in each case what is being gained by the 
adaptation is increased dynamic range rather than an inherent signal- 
to-noise ratio advantage over a nonadaptive technique. The adaptive 
techniques presuppose that the input signal variance is unknown. The 
quantizer step-size cannot therefore be meaningfully preset to an 
optimized constant value, but must be allowed to adapt itself to signal 
statistics in a fashion determined by a (time-invariant) adaptation 
strategy. 

The specific quantizer configuration that we consider is characterized 
by a uniform spacing of nonzero output levels, and Fig. 1 shows a 
snapshot of the quantizer at sampling instant r for the example of 


OUTPUT (Y) 






mg 


N= nfo vio N|~ 
- 


INPUT (X) 


Fig. 1—Uniform quantizer with 8 levels (B = 3). 
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B = 8. The step-size A, is adapted, for every new input sample, by a 
factor depending only on the knowledge of which quantizer slot was 
occupied by the previous signal sample. More precisely, if the outputs 
of a B-bit quantizer (B > 1) are of the form 


Au 


Yu= Pus 


; Pee 13.2" 32" = T3 A, > 0, (1) 
the step-size A, is given by the previous step-size multiplied by a time- 
invariant function of the previous code-word magnitude |P,—1| : 


A, = A,-1'M(|Pr-1|). (2) 


Note that, according to (2), the entire quantizer is ‘‘accordioned in’’ 
when M < 1 and stretched out when M > 1. The resulting quantizer 
is also uniform, with a step-size or “‘slot width” equal to A,. Practical 
implementations will also include upper and lower limits Amax and 
Amun for A,. This is discussed later in the paper. 

The above logic has been recently employed! for efficient differential 
encoding of speech signals at bit rates of 20 to 30 kb/s. The adaptation 
strategy (2) is indeed arbitrary.* But it represents, in the manner of the 
adaptive delta modulator discussed earlier by the author,’ a very 
simple, yet nontrivial, type of exponential adaptation, and sets a lower 
bound on the performance of possible sophistications that may include 
nonexponential adaptations and the use of longer word memories, L.e., 
the use of P,_2, P+—s, ete. 

An interesting result of this paper is that, for many interesting input 
signals, the step-size multiplier function M(|P|) which minimizes the 
mean-squared quantization error has the interesting property that it 
demands step-size decreases significantly slower than step-size in- 
creases. This is shown to be true for illustrative speech and picture 
signals and for first-order Gauss-Markovian inputs where the magni- 
tude of the correlation between adjacent signals is not too high (say, 
less than 0.9). 

In Section II, we discuss computer simulations with a first-order 
Gauss-Markov input. We discuss the simple case of a white signal 
(C = 0) at length. Results show the dependence of signal-to-quantiza- 
tion-error ratio (SNR) on the function M(|P|) for different values of 
B (number of quantizer bits), N (number of samples in input sequence), 
and Asrarr (initial step-size). We then specify adequate ranges of 


* Schlink has recently described another useful, but perhaps less general, empirical 
system.® Here, the adaptation consists in switching between only two quantizing 
characteristics. 
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variation for N and Asgrarzt, and thence determine a stable multiplier 
function that is optimal for a white Gaussian signal. Further results 
include the cases of C = 0.5 and 0.99, and show, for B = 2 and 3, 
values of Mopr and SNR gain over a nonadaptive quantizer. We also 
provide illustrative histograms of slot occupancies and observed step- 
sizes and a family of companding curves for a 4-bit quantizer. 

In Section ITI, we derive optimum multipliers theoretically for the 
examples of C = 0 and C—1. Results substantiate the values of 
Mopr from the computer simulation. We also compare our technique 
with that of Stroh? and discuss the greater efficacy of our adaptation 
strategy using the example of C = 0. Finally, in Section III, we discuss 
quantizer simulations with speech and picture inputs. We present 
multiplier functions basically similar to those for Gauss-Markov 
inputs. Optimal multipliers are found to be slightly different for PCM 
and DPCM coders. In the case of speech, we provide separate tables of 
Morr for B = 2; 3, 4, and 5. 


II. GAUSS-MARKOV INPUTS 


Our simulations have employed, as quantizer input, a first-order 
Gauss-Markovian sequence {X,} of 10,000 samples generated by the 
recursive rule 


Xy=C-Xuitvi-C-N,; Xo =0, (3) 


where the samples N, are drawn from a zero-mean, unit variance, 
white Gaussian sequence that is independent of past values of {X,}. 
The input sequence generated in (8) is itself Gaussian with a mean of 
zero, a variance of unity, and a correlation between adjacent samples 
equal to the preset constant C. 

The quantizer output, by definition, is the output level nearest to 
the input X,. It is formally written as 


7 xX, A ; xX, B-1 
ve={(2] 32] +1)3} sen x, A <2 


A X 
— B_ oe . “rT > OB-1 
{@"—1 Shem X,; Ate a (4) 
where [.- ] stands for “greatest integer in.”’ 
The quantization error 
E,=Y,—X, (5) 


has a magnitude that is bounded by A/2 except during overload which 
is expressed by the second line in eq. (4). 
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A conventional performance measure is the signal-to-quantization- 
error ratio 





dees 
SNR = Se? (6) 
where summations are assumed to be over the duration of a statistically 
adequate input sequence. 

We also refer in this paper to nonadaptive quantizers for which 


M(|P.al)=1; 9 all Pra 


Apa AS all r, 7) 


and the variation of signal-to-quantization-error ratio SNRwa is a 
function of the constant step-size A for this case. The step-size which 
maximizes SNRwna for a nonadaptive quantizer will be referred to as 
the optimum step-size Aopr. Values of Aopr and the corresponding 
values of SNRwa, for different values of B, have in fact been tabulated 
by Max’ for the case of C = 0. Max’s results also specify (via the 
Gaussian probability density function) the probability P, that the 
sth slot is occupied in an optimized nonadaptive quantizer: 


P, = Prob (Pu = 2s — 1) + Prob (—P. = 2s — 1); (8) 
Sy = LZ) eg 2 


where P,, is defined by (1). We will see presently that the probability 


P, is also very relevant in the study of an adaptive quantizer when 
C=0. 


2.1 A General Performance Criterion 


Adaptive quantizers are needed, as mentioned earlier, when non- 
stationary input signals are expected. Our simulations with Gaussian 
signals utilized a stationary input (3). To make the study of adaptation 
strategies meaningful in this stationary environment, we shall introduce 
some unconventional performance measures. For example, consider 
the ratio 


N N 
SNR(N, Asrart) = 2X a] E%, (9) 


where summations are over the first N samples of the input sequence. 
The dependence of SNR on Asrarr is significant only for small values 
of N. For large N, (9) tends to an asymptotic value that is independent 
of Agrart: 


SNR(#) & lim SNR(N, Asrart). (10) 
N 00 
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Fig. 2—Step-size histograms (B = 3, C = 0.5, N = 10,000). 


In fact, if N is sufficiently large, the value of Asrarr is entirely 
academic in the study of adaptive quantizers. See the step-size histo- 
grams in Fig. 2, for example. Notice how they are independent of 
Asrart, except for the flat tails representing transient values of A. 

In adaptive quantization, a suitable multiplier function for a given 
signal should provide a compromise between quickness of response [as 
measured by the magnitude of (9) for small values of N and bad values 
of Asrarr | and satisfactory steady-state performance [as measured by 
the magnitude of (9) for large values of N and values of Asrarr close 
to Aorr]. With these opposing factors in mind, we define an average 
performance index 


SNRave = = 5 > SNR, Asrtarr) (11) 


ASTART 


for values of NV = 10, 100, 1000, and 10,000, and 


Asrart = Fa Tag? b 1, V10, 10 | Kea 


The target values of N and Asrart above have been chosen with the 
following factors in mind: 


(z) First, as mentioned earlier, infinitesimally small ranges of 
values (for example, Asrarnt & Aopr; any N) are uninteresting 
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because they can result in Mopr values arbitrarily close to the 
trivial value of unity. 

(22) On the other hand, overly wide ranges of parameters which 
include combinations like (VN = 1, Asrarr = 10*Aopz) reflect 
pathological situations and lead to multiplier specifications that 
tend to be quite uncorrelated with the statistical nature of the 
signal being quantized. 

(777) As long as the extreme situations in (2) and (27) are avoided, it 
has been found that Morr values are not overly sensitive to the 
actual N and Asrarr values employed in the performance crite- 
rion (11), but depend mainly on the statistics of the signal being 
encoded. In fact, optimal multipliers in this case are merely the 
best multipliers in a variance-estimating problem (see the 
theory for C = 0 in Section III) that includes neither N nor 
Agrarr as a significant parameter. 

(iv) With the aforementioned factors in mind, the specific values 
of N and Agrarr in (11) were selected to have the following 
significance for a typical application such as speech quantiza- 
tion. First, the 40-dB range for Asrarr reflects an extent of 
uncertainty (about signal power) which is reasonably charac- 
teristic of telephone conversation.’ Second, when one considers 
Nyquist-sampled speech for applications like adaptive PCM or 
adaptive DPCM,! the values of N in (11) correspond at the 
lower end to about 1 millisecond of speech, and at the higher 
end to about 1 second of speech. This range clearly includes the 
range of durations that one may associate with ‘‘steady-state”’ 
or ‘‘stationary”’ segments in the acoustic waveform. In fact, if 
one considers phoneme durations, values of N in the range 100 
to 5000 seem to provide an adequate model. It is our contention 
that by using N values of this type in an index of performance 
such as (11), we can very usefully assess M-functions for 
quantizing locally stationary signals such as speech, even when 
simulating the quantizer with a (standard and easily dupli- 
cated) stationary Gaussian input. Actually, however, we have 
carried out completely independent simulations with real speceh 
signals as well (Section IV), and the results of this section are 
directed toward the quantization of Gaussian inputs as such. 


2.2 Multiplier Functions for B = 2, C = 0 


Table I illustrates the nature of the SNR function (9) for two 
multiplier functions in a 2-bit quantizer. The first multiplier function 
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TABLE J—EXAmPLe or SNR Functions ror B = 2, C = 0 
(ENTRIES IN dB) 





Values of NV 
20 log (Seusr) 10 100 1000 10,000 
OPT 
M, = 0.8 M2 = 1.6 

—20 6.4 7.2 74 7.3 
~10 10:5 8.9 79 73 
0 9.7 8.3 7.6 73 
10 5.8 7.2 74 72 
20 —5.9 4.9 71 73 

M, = 0.98 Mz = 1.04 
—20 1.6 3.8 8.4 9.1 
~10 5.2 5.8 8.9 9.2 
0 10.7 8.0 9.4 9.2 
10 0.0 5.9 9.0 9.2 
20 ~13.2 ~5.0 3.5 8.1 





shows quicker response (better SNR values for.V = 10 and 100), while 
the second function achieves a better asymptotic value of SNR (at 
N = 10,000). Obviously, the poor asymptotic performance of the first 
M-function is due to overly abrupt step-size oscillations in the “‘steady- 
state,” while the inferior performance of the second M-function for 
small N is due to sluggish adaptations of A when Asrarr is suboptimal. 

Table II compares several /-functions* for a 2-bit quantizer on the 
basis of (11). The functions included represent a subset of many more 
functions which were simulated and compared on the basis of SNRave. 
The best value of 6.8 dB has been noted for M, = 0.80, M2 = 1.60, 
although this function provides a clearly nonmaximal asymptotic per- 
formance (Table I). The first five functions in Table II also satisfy 


TaBLEe I]—Comparison or MuutTipLieR Functions (B = 2, C = 0) 


M, Mz SNRave (dB) 
0.71 2.00 5.9 
0.80 1.60 6.8 
0.90 1.20 6.5 
0.95 1.10 6.1 
0.98 1.04 5.3 
0.95 1.20 5.9 
0.50 2.00 5.8 
0.90 1.10 5.2 





* Whenever there is no scope for confusion, we shall use the symbols M,, M2, Ms, 
and M, instead of M,(1), M,(3), (5), and M,(7). 
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the interesting constraint suggested by Goodman :! 

Mj: M2 = MY" Me = MP MP = 1, (12) 
where P; 0.67 and P. 0.33 are the probabilities of inner- and 
outer-slot occupancy in a nonadaptive quantizer with an optimal 
Aorr for the Gaussian input. Goodman conjectures that the prob- 
abilities of using Mi and M, in a well-designed adaptive quantizer 
should indeed be equal to the parameters P; and P2 of the nonadaptive 
quantizer. A constraint of the form (12) then represents a stability 

0.6 
0.5 
0.4 
0.3 


0.2 


0.1 





M2= 2.00 


PROBABILITY 





-16 —12 -8 


10 LOG A, 


Fig. 3—Step-size histograms (B = 2, C = 0, N = 10,000). 
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SNR IN dB 





10 100 1000 10,000 
N 


Fig. 4—Comparison of multiplier functions (B = 2, C = 0, Asrart = 0.1 Aopr). 


criterion which specifies that the random step-size A, neither grows out 
of bounds, independently of the input, nor decays to infinitesimal 
values. This criterion has been discussed earlier in the context of 
adaptive delta modulation with a 1-bit memory.® 

The desirability of constraint (12) on step-size multipliers is also 
demonstrated by the step-size histograms in Fig. 3. The multiplier 
pairs (0.9, 1.2) and (0.71, 2.0) satisfy constraint (12), and the corre- 
sponding histograms have the desirable property that they are centered 
on Aopr although they have different dispersions (suggesting differences 
in quickness of response and steady-state performance). The function 
(0.9, 1.10), on the other hand, produces a histogram whose mode is 
clearly displaced from Aopr. This suggests that (0.9, 1.10) falls in a 
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TasBLeE IJI—SNR Function ror M, = 0.90, Mz = 0.90, M3 = 1.25, 
M, = 1.75 (B = 3, C = 0, Entriss 1n dB) 





i ewan Values of NV 
20 log (As) 
Aopt 10 100 1000 10,000 
—20 11.9 11.2 12.7 12.7 
—10 14.5 11.8 12.6 12.7 
0 15.7 11.2 12.9 12.7 
+10 11.8 11.6 12.5 12.7 
+20 —1.6 8.7 12.3 12.7 


class of inefficient multiplier functions; this is attributed to the fact 
that the function (0.9, 1.10) clearly violates requirement (12) above.!! 

Finally, in Fig. 4, we show SNR (6) as a function of N for a fixed 
value of Asrarr, and for different M-functions. It is once again ap- 
parent that the adaptation function (0.8, 1.6) provides an attractive 
combination of responsiveness and asymptotic performance for B = 2. 


2.3 Multiplier Functions for B = 3, C = 0 


Table III demonstrates the nature of the SNR function (9) for 
B = 38and aspecific multiplier function. Table IV uses the performance 
criterion (11) to show the efficiency of this multiplier function (0.9, 
0.9, 1.25, 1.75). As in the 2-bit example, the M-functions in Table IV 
are only a subset of a much larger set of M-functions which were 
simulated and compared on the basis of SNRave. We have only 
included the most interesting functions from our search for maximum 
SNRave. The first three M-functions in Table IV satisfy a stability 
constraint analogous to (11): 


M9-4917931 9-16 179-07 = MP. ME? MPO. MP = 1, (13) 


It is interesting that the best function in Table IV belongs to the class 
of functions obeying (13). Notice also that the reduction of the number 
of distinct step-size multipliers (second row in Table IV) leads to a 


TABLE IV—ComPaRISON OF MULTIPLIER Functions (B = 3, C = 0) 


My; M, M; M, SNRave (dB) 
0.90 0.90 1.25 1.75 11.7 
0.90 1.00 1.00 75 11.4 
0.5 1.0 1.0 2.0 9.6 
0:3 0.9 5 21 8.9 
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PROBABILITY 





SLOT NUMBER 
Fig. 5—Histogram of slot occupancies (B = 3, C = 0, N = 10,000). 


marginal decrease of SNRavz. This tolerance to a reduction of the 
number of distinct multipliers does seem to extend, although in lesser 
measure, to larger values of B and to speech and picture signals. 

Finally, Fig. 5 shows a histogram of slot occupancies for the best 
M-function in Table IV. The number of quantizer slots or output 
levels is equal to four (neglecting signs), and the dotted fifth slot 
refers to the overload probability that has been accumulated into the 
fourth bar of the histogram. It is interesting that, despite step-size 
adaptations, the Gaussian nature of the input density function shows 
up in the histogram. The heights of the bars in Fig. 5 represent experi- 
mental slot probabilities of 0.47, 0.30, 0.14, and 0.09. Notice again 
that, in the manner of (138): 


0).99-47 .,.99-30. 1 250-14. 1750-09 = 0.9904 ~ 1. (14) 


2.4 Comparison of Adaptive and Nonadaptive Quantizers 


Table V summarizes the nature of optimal multiplier functions for 
B = 2 and 3. These functions are obtained on the basis of criterion 
(11). Values of M are generally rounded, representing broad optima, 
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TABLE V—QUANTIZATION OF GAuss-Markov Inputs [ENTRIES ARE 
SNR (10,000, Aopr) VALUES IN dB] 


B C 0.00 0.50 0.99 
SNRwa 9 9 9 
M (1) 0.8 0.8 0.5 
M (2) 1.6 1.6 2.0 
SNRwa 14 14 14 
SNRa 13 13 16 

3 M (1) 0.90 0.90 0.30 

; M (2) 0.90 0.90 0.90 
M (8) 1.25 1.25 1.50 
M (4) 1.75 1.75 2.10 


and the precision in the specification of M values may be as bad as 
+5 percent in some cases. 

To provide a fair comparison with optimal nonadaptive quantizers, 
the performance figure used in Table V is the asymptotic value (10). 
Formally, the notation used in the table is as follows: 


SNR = SNR(10,000, Aopr). (15) 


The subscript A refers to the adaptive quantizer with step-size multi- 
pliers optimized using (11), while the subscript NA refers to a non- 


P(X; |X-—-1) 





Fig. 6—Conditional density function of quantizer input. 
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adaptive quantizer with constant step-size Aopr. The SNR values are 
in dB, and are rounded to the nearest integer. 

Note that negative values of C are not included in the table. The 
assumption of a symmetrical quantizer (Fig. 1) renders the quantizer 
design independent of the sign of C. Specifically, the quantizer input 
X, has a probability density function (conditioned to X,_1) that is 
sketched in Fig. 6; the optimum step-size is that which fits the 
quantizer to this density function in a way that minimizes the sum of 
overload error variance and granular error power. This optimum 
depends only on the disposition of the PDF in Fig. 6 and the magnitude 
of the nonzero mean, not the sign of it. 

Finally, Table V assumes that no constraints exist on the minimum 
and maximum values of step-size. Practical implementations will, of 
course, involve such constraints (see Fig. 8), as well as constraints on 
actual multiplier values. Significant conclusions from Table V are the 
following: 

(2) Except for C = 0.99, optimal multipliers are such that step-size 
decreases are always slower than step-size increases. The observation 
has been found to extend for B = 4 also and, as seen later (Section IV), 
to the quantization of speech and picture signals as well. 

The need for fast increases of step-size and slow decreases thereof 
may be physically explained as follows. Quantization errors during 
overload tend to be more harmful than those during granularity, in 
that the magnitude of granular error is restricted, by definition, to a 
half step-size, while no such simple constraint exists for an overload 
error. It is therefore reasonable to decrease step-sizes (relatively) 
slowly to avoid unduly small step-sizes leading to the harmful overload 
errors. The observation is obviously less significant for a coarser 
quantizer than for a finer quantizer because granular errors in the 
former are more comparable in magnitude to overload errors and 
hence more equally harmful. This is indeed reflected in Table V. Note 
that, for a given value of C, the disparity in rates of step-size increases 
and step-size decreases is least for the coarser quantizer (B = 2). 

There is an alternative explanation for (z) above, which also clarifies 
why the disparity between the speeds of step-size increase and step- 
size decrease is less apparent for large values of C’. Refer to the stability 
constraints (12) and (18), as discussed for the case of C = 0. It turns 
out that in the uniform (nonadaptive) quantization of a Gaussian 
signal, the probability P, (8) is a monotonically decreasing function of 
s. It follows then, as seen in (12) and (13), that multipliers for step- 
size decreases have greater probabilities of being employed, and hence 
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must lead to slower step-size changes each time they are actually used. 
Explicitly, for B = 2, (12) can be rewritten: 


(nM, _ Pi 
in(i/M) Ps a 


Obviously, then, if P1 > Pe, the step-size increase (as given by M>») 
is faster than the step-size decrease (as given by M/)). 

The argument for nonzero values of C is very similar, except that 
the probabilities P, peculiar to a Gaussian probability density function 
should now be replaced by probabilities P,(C) that refer to the uniform 
quantization of the asymmetrical conditional PDF in Fig. 6. Ap- 
parently, the probabilities P,(C) are not monotonically decreasing for 
C = 0.99. This is why the requirement of relatively more rapid step- 
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Fig. 7—Histogram of slot occupancies (B = 3, C = 0.99, N = 10,000). 
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size increases is waived for the example of C = 0.99 (while being still 
true for C = 0.5). 

Figure 7 shows a histogram of slot-occupancies for B = 3 and 
C = 0.99. Compare this non-monotonic PDF with the Gaussian histo- 
gram for C = 0 (Fig. 5). In analogy with (13), the stability criterion 
associated with Fig. 7 is 


0.3°-97 0.9-6 150-71 2,19-09 = 0.9938 = 1. (17) 


Finally, it should be mentioned that the (relatively) slow step-size 
decreases in Table V are fast enough, in an absolute sense, for typical 
quantizer applications. For example, if M = 0.95, and a step-size 
decrease of 20 dB is needed for adaptation to an idle-channel situation 
in speech quantization, the time needed for such adaptation will be 
45 samples. For Nyquist-sampled speech, this is only about 5 ms. 

(zz) Although the quantization problem for C = 0.5 is qualitatively 
similar to that for C = 0.99 (Fig. 6), we note that results for C = 0.5 
(Table V) are nearly identical with those for C = 0. The differences in 
Morr values that are caused by a nonzero C' = 0.5 were apparently 
too small to be detected in our finite search for best multipliers. 

(i727) Referring again to Table V, the best adaptive quantizers seem 
to have an SNR advantage over the nonadaptive scheme (working 
with an optimal step-size) only for very highly correlated inputs. In 
fact, in many instances, the SNR gain resulting from adaptation is 
seen to be negative (due, evidently, to overly abrupt manipulations of 
step-size). 


The reason for using an adaptive quantizer in these situations is 
only to facilitate quantizations with much less knowledge of the input— 
equivalently, with much less knowledge of Aopr than is necessary for 
an equivalent performance in the nonadaptive case. In other words, 
step-size adaptions increase the dynamic range of the quantizer and 
enable it to handle inputs with large amplitude variations, such as 
nonstationary signals. 

The above idea has already been demonstrated by the asymptotic 
SNR values in Tables I and III. To provide a more application- 
oriented illustration, we undertook two extensions of our computer 
simulation. These experiments employed B = 4, C = 0.5, and the fol- 
lowing multiplier function: 


(0.90, 0.90, 0.95, 1.0, 1.2, 1.5, 1.8, 2.1). (18) 


Finite step-size dictionaries were used, determined by maximum and 
minimum step-sizes Amax and Amr. The starting step-size was set 
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equal to Aopr, subject, however, to modification because of the con- 
straints Amax and Amin. 

In the first of these extensions, the step-size dictionary had the 
characterization 


Amax: Amin = AGpr (19) 
Amax/Amin = R (20) 


and the quantizer performance was studied in terms of SNR (10,000, 
Aorr) as a function of R. It was reassuring to note that the SNR was 
constant to within 1 dB for sample values of FR in the range 1 to 
o—due, no doubt, to the safe design feature (19). In fact, a maximum 
SNR was noted for a noninfinite value of R. 

In a more revealing second experiment, the quantizer was “‘centered”’ 
at a value Amyp not necessarily equal to Aopr: 


AmaxAmin = Abr (21) 


and the performance was measured as a function of Ayyp for values of 
R(20) equal to 1, 10, and 100. Note that R = 1 refers to the non- 
adaptive case. 

Figure 8 plots these results. The monotonic improvement of dynamic 
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Fig. 8—Companding characteristics (B = 4, C = 0.5). 
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range with increasing R is apparent. It is expected! that practical 
quantizers can be designed with values of FR equal to 100 or more. 


III. THEORETICAL DERIVATION OF OPTIMAL MULTIPLIERS 


In this section, we shall regard the adaptive quantization problem 
as one of learning signal variance. In other words, the problem of 
determining an optimum instantaneous step-size A, is regarded as 
being tantamount to that of finding the best estimate at time r of 
the conditional standard deviation S, of the quantizer input; and of 
setting A, proportional to this estimate: 


AOPT = K(B)-S,(A,1, Py-1). (22) 
3.1 Case of C = 0 


The constant K is an obvious function of the number of quantizer 
levels and, hence, of B. For the problem of uniform quantization of a 
zero-mean Gaussian signal, Max’s Table II" specifies the following 
values for K(B) :** 
K(1) = 1.596, K(2) = 0.996, 
K(3) = 0.586, K(4) = 0.335. 
The dependence of S$, on A,_1 and P,_1 (22) is, of course, characteristic 
of an adaptation strategy which uses a 1-word memory. 
We now propose that the variance of X, be estimated as the average 
of the squares of (7) X,_1, the most recent quantizer input, and (72) 
S,-1, the most recent sr of S. In other words, let 


St = 4(X?_, + $2.,).1 (24) 
We next recall the identity 
Xe = Vea _— Hy a 


(23) 


Pe Gass 
2 


where H,_1 is the quantization error. Furthermore, by virtue of the 
basic algorithm (22), we suggest that 


Ors = A;_1/K? (26) 
Let us use (25) and (26) in (24) and set the resulting value of 5S, in 
(22). We obtain, after some Soule 


2 
(A2?T/A,_1)? = . | = iis i + Aa (Gia — 1 — Bead sPr) | - (27) 


* These K values are relevant for C = 0 because, in this case, the conditional 
conslty function (Fig. 6) is indeed zero-mean Gaussian. 

*In general, one may consider a weighted average of the type uaz? + vs*. The 
case of u = 0 will be appropriate for “steady-state’’ operation, and the use of v = 0 
will be appropriate for a “transient” situation. The need for time-invariant step-size 
saa ad suggests a compromise design characterized by a weighting of the type 
u=v = 0.5. 


ae er (25) 
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E,-1 is an unknown random variable, but the following can be said 
about its role in (27): 

First, the £?_, term is significant only for the last quantizer slot in 
which, due to possible overload, E?_, can be arbitrarily large. Further- 
more, for this end slot the —#,1A,_1P,_1 term tends to be positive. 
Notice, from definition (25), HE is negative in overload when P is 
positive and vice versa. 

For the remaining quantizer slots, H?_, is again positive but no 
longer significant, and —H,_,A,_1P,_1 is expected to be negligible as 
well, on the average. This is by virtue of the uniform PDF approxi- 
mation for granular errors 


P(E) =1/d; 5 < Eu <5 (28) 

and a consequent decorrelation of output PA and error E. 
The optimum multiplier function [square root of (27) ] can therefore 
be expressed in the form 

L. de 3 
mer = [5+ 5 PP +eIPal; €=0; 29) 
where 6 is a positive correction term that is significant only for the 
end slot: 


&(|P,1|) £0 if [Pre 2B — 1, (30) 
Table VI compares the M values from (29) with those from the 
simulation in Section II. 
3.2 Comparison With Stroh’s Adaptation Logic (C = 0) 


Consider, in place of (24), a simpler variance estimation of the 
type considered by Stroh ? 


Ss? = ae (31) 


This results in, by virtue of (25), (22), and arguments similar to those 
at the end of the previous paragraph, a multiplier function of the form 


M, = = [Poal| + 8(|Pral); 
(32) 
8(|Pxa|) 20 if [Pal #2? —1. 


Table VI lists values of MPP? (29), M, (32), and the experimental 
optima Mexp from Table V. Values of K have been taken from (23). 
Notice how MP??? provides a better specification of optimal multi- 
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TABLE VI—CoMPARISON OF MULTIPLIER FUNCTIONS 








B=2 B=3 
M, Mprt Mexr |P,-1| M, fe Mexp 
0.50 0.79 0.80 1 0.29 0.75 0.90 
1.50 + & 1.27 + & 1.60 2 0.87 0.94 0.90 
— — _ 3 1.45 1.26 1.25 
— —_ — 4 2.00 + 6 1.61 + & 1.75 


pliers than does M,. Furthermore, as B increases, the constant K 
approaches zero and the theoretical multiplier functions for the inner- 
most slot (P;-1 = +1) have the following limiting behavior: 


lim M,(1) = 0 (33) 
Boo 
lim M°PT(1) = Vi/2 = 0.71. (34) 
Bow 


Simulations with B = 4 and 5 have verified that the trend in (84) 
is indeed more realistic than that in (33). 

It should be mentioned that the adaptation strategy (31) is only 
the simplest case of Stroh’s? method which has a general variance 
estimator of the form 

a 1 2 

Sin = — L Xiu. (35) 

Nu=1 

It is interesting, nevertheless, that for the same length (n = 1 or 
one-word) of quantizer memory, our adaptation rule specifies better 
step-size multipliers, as seen in Table V. In fact, the use of MP??* 
yields for (B = 3, C = 0), an SNR (for Gaussian signals) which is 
better than what Stroh reports for n = 2 (10 dB; N = 2500) in his 
Fig. 3.38. With the experimentally optimized M-function (Table V), we 
indeed do significantly better and the SNR value of 12.7 dB for this 
case is equivalent to n = 6 in Stroh’s logic and falls short of the 
optimum (n = © in Stroh) by not much more than 1 dB. 

The efficiency of our logic is clearly attributable to the way we 
exploit quantizer memory, namely, in terms of P and A, rather than 
in terms of the product of the two quantities (the quantizer output Y 
used by Stroh). Physically, the use of PA for adaptation seems to 
wipe out some of the “‘overload”’ and “underload” cues that an indi- 
vidual knowledge of P and A preserves. 
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TABLE VII—Comparison OF THEORETICAL(M??T) AND EXpPERI- 
MENTAL (MF*?, In PARENTHESES) MULTIPLIERS 


B C 0 0.99 
9 M (1) 0.79 (0.8) 0.5 (0.5) 
M (2) [1.27 + 8](1.6) [1.5 + 8 ](2.0) 
M (1) 0.75 (0.9) 0.25 (0.3) 
3 M (2) 0.94.(0.9) 0.75 (0.9) 
M (3) 1.26 (1.25) 1.25 (1.5) 
M (4) [1.61 + 8? ](1.75) [1.75 + 8] (2.1) 


3.3 Case of C1 


When the adjacent signal correlation C approaches unity, the con- 
ditional PDF (probability density function) of X, approaches a 
Gaussian spike centered at CX,1 (Fig. 6). The width of the spike 
is proportional to the square root of (1 — C*), and therefore approaches 
zero irrespective of the value of signal variance S. The adaptive 
quantization problem is no longer one of variance estimation. It will 
consist, instead, in a ‘‘fool-proof”’ strategy of the following type: 
Select a step-size A, such that the PDF spike at CX;_, falls right in 
the middle of the positive (or negative) half of the quantizer range, 
assuming that CX,-1 is positive (or negative). If we recall that a 
B-bit quantizer has a half-range width equal to 22—-!A, we see the 
requirement (assuming positive quantities throughout) is: 


QB-1A, 
2 ? 





CX 1 = C-1. (36) 
The logic clearly provides simultaneous protection against both over- 
load and underload. Utilizing the estimate of X,_1 (25) in (386), we 
obtain the condition 





C | = E,-.| =A, C1, (37) 
Equivalently, with usual assumptions on the quantization error E,_1, 
P A, |P,-1| 
OPT — — 2 
eae Ba at (Peal) (38) 


8(|Pral) 20 if P,1¥23—1. 


* See the spike in the histogram of Fig. 7. 
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3.4 Comparison with Simulation Results 


Results for a general value of C(O < C < 1) can in principle be 
attempted on the basis of a general PDF such as Fig. 6. However, 
tractable derivations seem to require too many simplifying assumptions 
to make the theory worthwhile, especially in view of the observation 
(Table V) that the correlation becomes significant only if C—1. We 
therefore conclude this section by merely listing, in Table VII, theo- 
retical step-size multipliers for C = 0 and 0.99 [from (29) and (88) ] 
together with the experimentally optimized multipliers from Section II. 


IV. QUANTIZER SIMULATIONS WITH SPEECH AND PICTURE SIGNALS 


In this section, we present results from computer simulations of the 
adaptive quantizer with speech and picture inputs. 

The results in Table VIII refer to a low-pass-filtered speech signal 
(about a second long), and a single frame of picture input (the face of 
Karen in Picturephone® format®). Listed are step-size multipliers found, 
by search procedure, to maximize an asymptotic SNR (10) as measured 
over the entire length (N > 10,000) of the input sequences. 

The following observations are of interest: 


(t) The signal PDF seems to have a significant effect (presum- 
ably through overload statistics and the end-slot correction 
6*(|P,-1|) of Section III) on the largest step-size multiplier. 
Note the value of M, for picture input. 


TaBLE VIJJ—Srep-Size MULTIPLIERS FOR ILLUSTRATIVE SPEECH AND 
PicTURE SIGNALS (ENTRIES IN PARENTHESES REFER TO PICTURES) 


BN Coder 


Type PCM DPCM 

2 0.6, 2.2 0.8, 1.6 

3 0.85, 1, 1, 1.5 0.9, 0.9, 1.25, 1.75 
(0.9, 0.95, 1.5, 2.5) (0.9, 0.95, 1.5, 2.75) 

4 0.8, 0.8, 0.8, 0.8, 0.9, 0.9, 0.9, 0.9, 
1.2, 1.6, 2.0, 2.4 1.2, 1.6, 2.0, 2.4 

5 0.85, 0.85, 0.85, 0.85, 0.9, 0.9, 0.9, 0.9, 
0.85, 0.85, 0.85, 0.85, 0.95, 0.95, 0.95, 0.95, 
1.2, 1.4, 1.6, 1.8, 1.2, 1.5, 1.8, 2.1, 
2.0, 2.2, 2.4, 2.6 2.4, 2.7, 3.0, 3.3 


“ 
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1 <r 


ANI 


[Pr 
28-1 





Fig. 9—Desirable form of the multiplier function M for the adaptive quantization 
of speech signals (B > 2) and first-order Gauss-Markov signals which are not highly 
correlated (say, C > 0.5). 


(iz) Differentiation has the effect of decreasing adjacent sample 
correlation. This seems to explain differences in multipliers as 
applied to PCM and differential PCM quantizers for speech. 
Note that the effect is most pronounced for B = 2. 

(az) Although the input signals are not first-order Markovian, the 
multipliers have the earlier-mentioned property that step-size 
increases are relatively more rapid than step-size decreases. 
Refer to the general diagram in Fig. 9. 
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TABLE IX—CoMPARISON OF SPEECH QUANTIZERS 
(ENTRIES ARE SNR VALUEs IN dB) 





Logarithmic Adaptive Adaptive Adaptive 
PCM with PCM with DPCM with DPCM with 
p-law Uniform Uniform Nonuniform 
B Quantization Quantization Quantization Quantization 
2 3 9 13 12 
3 8 15 18 18 
4 15 19 22 24 


It may be mentioned that in each of the above simulations, the 
adaptive techniques also registered an SNR gain of 2 to 4 dB over 
optimized nonadaptive quantizers. Table 1X shows some results per- 
taining to a band-pass-filtered speech sample. These results were 
obtained from an independent experiment on coder assessment.” The 
adaptive quantizers (APCM, ADPCM) used the multipliers of Table 
VIII*, and the nonuniform quantizer characteristics employed in 
adaptive DPCM are those recommended by Paez and Glisson.® 
Finally, the log-PCM used a u» = 100,’ and the adaptive quantizers used 
a Maximum-to-minimum-step-size ratio of 100. 

Notice from the table that adaptive quantization, as incorporated 
into PCM, has the potential of outperforming the conventional 
technique of logarithmic companding. Evidently the advantages over 
log-PCM are even more impressive in ADPCM, and a companion 
paper will discuss, at length, the use of 3-bit and 4-bit adaptive 
quantizers in the DPCM coding of speech. 
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Heuristic Solution of a Signal 
Design Optimization Problem 


By B. W. KERNIGHAN and S. LIN 
(Manuscript received February 15, 1973) 


This paper discusses a heuristic solution procedure for a combinatorial 
optimization problem that originates in designing signal constellations 
for modems. 

The design problem is to place m signals in a two-dimensional space 
to minimize the average error rate under specified noise conditions, using 
a maximum-likelihood decoding scheme. Intuitively, it amounts (roughly) 
to spreading the signal points as far apart as possible, according to the 
distance measurement implied by the noise function. 

We show how this problem can be reduced to a discrete one: Given an 
£ by n matrix P, and m < #, find an m-row subset M = {11, +--+ , im} of 
the rows of P that maximizes 


ye Max ij, 
j=l 1CM 
and then describe an efficient procedure for finding this maximizing set. 
Experiments indicate that the procedure is a useful tool, both for 
analysis of existing and proposed signal constellations and for finding 
new, near-optimum ones. 


I. THE PHYSICAL PROBLEM 


This paper discusses a heuristic procedure for solving a combinatorial 
optimization problem that arises in designing signal constellations for 
modems. The solution method is also applicable to the covering 
problem; we will discuss this at the end of this section. 

The underlying physical problem is the following: A digital signal 
s is to be sent through a noisy channel. In general, s may take on only 
a finite number, m, of distinct values. (In practice, m will be a power 
of 2.) Since the transmission line is an analog device, any specific s 
value is encoded for transmission by modulating a carrier wave for a 
period of time. For instance, s might take on only the two values 0 and 

1145 
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1, in which case the modulation might be to send either of two ampli- 
tudes (amplitude modulation) or to send either of two frequencies 
(frequency modulation). 

The modulation considered here is more complex: A specific value 
of s will be encoded as 


g(t)[a sin wt + b cos wet |, 


where w, is the carrier frequency, g is an appropriate pulse shape, and 
a and 6 are the amplitudes of the sine and cosine components. 

The signal s is quantized into one of m levels; for each level, there is 
a corresponding a and b, so there are m different (a, b) pairs. 

The received signal, as always, is corrupted by noise, so a sample 
value sent as (a, b) is received as (a’, b’). At the receiver, a decoder 
processes this corrupted (a’, b’) and attempts, according to some 
criterion for minimizing the error rate, to reconstruct the (a, b) which 
was sent originally. 

The design question is: What (a, b) pairs should be chosen to 
minimize the error rate for this decoding process? 

In the version of the problem we solve, the main constraint is that 
a? + b? is bounded for all (a, b) pairs, which implies that peak signal 
power is bounded. A related but more difficult problem requires that 
the average of a? + b? over the (a, b) pairs is constant; this corresponds 
to a bound on average power. We discuss this problem in Section VII. 

The combinatorial optimization problem is a discrete version of this 
design question. Let us replace the continuum of points that could 
represent (a, 6) values (everything inside the circle a? + b? = 1) by @ 
discrete points, spread more or less uniformly throughout the region. 
We call these “allowable” signal values. Since the noise may add to the 
received amplitude, the received signal can in fact be outside this 
circle. Let us define additional n — ¢ discrete points to represent the ad- 
ditional possible received signals that lie outside the circle a? + b? = 1. 

Now define an ¢ by n matrix P = {p,;} by 


pij = probability that, if signal 7 were sent, it would be 
received as (discrete) point 7 


t=1,---,% g=lereyn 


Suppose for convenience that the chosen signal values are points 1 to 
m (i.e., rows 1 through m of P). The decoding procedure to be used is 
simply this: If 7 is the received signal, it is decoded as that zin 1, --- ,m 
for which p;; is maximum. If 1, --- , m have equal a priort probabilities, 
this procedure minimizes the error probability. 
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The probability that a particular signal 7 is decoded correctly is the 
probability that it falls into a column j where it is the largest entry. 
The probability of correct decoding using any particular set of m rows 
of P is thus the sum of the column maximum elements in those m 
rows (divided by m). The problem (at last) is to find those m rows that 
maximize this probability; these will be the signals used. More for- 
mally, find an m-row subset M = {71, --+,%m} of the rows of P that 
maximizes 


Vu = dX max py. 
j=1 iGM 
We call Vy, the value of the subset M. 

In the physical problem, m < ¢ < n; as an abstract problem, the 
latter inequality is unnecessary. 

Note that the algorithm we will present is essentially insensitive to 
the characteristics of the matrix P. In practice, this means, for example, 
that any noise characteristics can be treated effectively. This includes 
not only the classical additive white noise, but also phase and ampli- 
tude jitter components. 

As an example of a matrix with quite different characteristics, 
suppose P has entries which are either 0 or 1. A covering problem is 
“Find a minimum set of rows of P such that these rows together con- 
tain a 1 in each column of P.’”’ We can use our heuristic procedure to 
find approximate solutions for the covering problem as follows: Find 
a maximum value solution of the original problem, using m rows. If 
the value is less than the number of columns of P, increase m; if it 
equals the number of columns, decrease m. Find a new solution with 
the new m. The smallest value of m for which the value equals the 
number of columns is a minimum cover of P. 


II, A HEURISTIC PROCEDURE 


The process is based on iterative improvement of random initial 
solutions. A random set of m rows is chosen from the ¢ possible rows. 
(In practice, of course, we can also let the procedure try to improve 
on a specific initial set.) We augment the m initial rows by one row 
chosen from the ¢ — m unused rows, giving us m + 1 rows. We then 
compute which of these m + 1 rows contributes the least to the value 
of the set and remove it. (The row removed might well be the row 
added.) We then move to the next row in the unused ones and add it 
to the current m rows. The process terminates when all the ¢— m 
currently unused rows have been examined without finding a profitable 
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replacement. This defines a local optimum solution. We then iterate 
the entire procedure from a new random start. 

The resulting solutions are ‘‘1-opt”’ in the sense of Reference 1—that 
is, no exehange of a single pair of rows can improve the solution. 
Although 1l-opt procedures are among the weaker heuristics, the 
results are quite acceptable, as we shall see in the next section. 

The process is very fast which counteracts the lower effectiveness 
of l-opting: a 100 by 100 problem takes about one second on the 
Honeywell 6070 (in FORTRAN A). The run time for dense matrices 
is essentially proportional to én and independent of m. For sparse 
matrices (the situation that occurs in practice), the run time varies 
only with the number of non-zero elements, which for real problems 
is proportional to fn. 

The process is fast, partly because care is taken to do no extra work. 
In detail, a basic step of the algorithm is as follows (the next section 
contains a numerical example, which can be followed in parallel) : 

Initialization. Suppose without loss of generality that the initial rows 
are 1, --- , m. Call this set M. Let v(z), 7 = 1, --+ , m, be the decrease 
in value if row 7 is removed from M. We will compute v. This is done 
only once per local optimum solution. 

We begin by setting v(-) = 0. Each column j (1 S$ 7 S n) con- 
tributes an increment to exactly one component of v, as follows. Find 
x; and y;, the largest and second-largest elements among the first m 
elements of column j. Record these, and also the rows in which they 
were found, 7, and 2, (1 S 2,, ty S m). Now, since 2; is the largest 
element in column j, it determines the contribution that column makes 
to the value of M. But if row 7, were removed from M, the contribution 
of column j would be determined by y;, the second largest element. 
Thus, the decrease in value that would result if row 7, were removed 
from M is x; — y;, so we add x; — y; to v(t). This process is done for 
each column. 

Phase 1—Evaluation of a Replacement Row. When the initialization is 
finished, we evaluate replacement rows. Suppose rowr(m +1 Sr S&S ?) 
is the next proposed replacement. We will compute which of the m + 1 
rows in M, = {1,---,m,r} decreases the value of M, least when 
removed. We will do this without examining any of the matrix P 
except for row r itself. 

Let A(z) be the change in v(z) that results if row 7 is removed from 
M,. By computing A, we do not need to change v unless we are actually 
going to exchange two rows. Initially, let A@) = 0,7 = 1, --- , m,r. 
(Let v(r) = 0 as well.) The value A(z) will be $0 for 7 in M while 
A(r) = 0. 
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For each column 7, let z = p,;;, © = x;, and y = y;; we will do one 

of (2), (22), or (zit): 

(1) If z S y, the new element is smaller than second best; no 
action is necessary. 

(ii) If y < 2S x, 21s a new second-best element. The A value for 
the row containing x, A(z), must be decreased by z — y, since 
the contribution to v(z,) from column j is now x — z instead 
of x — y. 

(tic) If 2 > x, we have a new largest element in the column. Add 
z — x to A(r) (x is now second largest) and subtract x — y from 
A(iz), since row 7, no longer contains the largest element in 
this column. 


Phase 2—Determination of Which Row to Remove. We have now 
determined A(z), the change in value that would result if row 7 were 
removed from M,. Find the minimum among o(1) + A(l), ---, 
v(m) + A(m), v(r) + Ar). 

If the minimum occurs at row k #7, let us say, then we must 
exchange rows r and k, update v(-), and update the records of largest 
and second-largest elements for each column. Go to Phase 3. 

If this minimum occurs at 7, there is no profit in replacing one of 
1, ---, m by r. If €— m rows have been consecutively examined 
without profit, we have finished; the set M is the local optimum solu- 
tion. Otherwise, we must set r to r + 1 (wrapping around from ¢ to 
m. + 1 if necessary) and go back to the Phase 1 calculation. 

Phase 8—Updating After Exchange of Two Rows. As in Phase 1, we 
must perform one of (2), (i), or (iz) below for each column. The 
element x is the largest in M, y is the second largest, and z is the new 
element from row r. Row k is the row being ejected (1 S$ k S$ m). 

Case (2):2 Sy. If k #7, and k ¥17,, then we are not replacing 
either of the two largest elements in this column, so no updating is 
necessary. Go to the next column. 

If k = 7,, we are replacing the largest element with something no 
better than third largest. We replace x by y (and 7, by 7,) and find a 
new number two element in M@,—call it w. Since y is now largest, we 
subtract w — y from v(zy) and then let i, = iy. 

If k = z,, we are replacing the number two element with something 
no better than third largest. Again we search for w, the new number 
two, update v(7,) by subtracting w — y, and update 2,. 

Case (11): y < 2S x. If k = 7,, we are replacing the largest element 
with a new and smaller largest element. The element z replaces x as 
the largest element, and A(r) is increased by z — y. 
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If k #7i,, 2 becomes the new number two element, and z — y is 
subtracted from v(z,) to reflect the smaller difference between first and 
second elements. 

Case (i171): « < z. If k = 7,, we are replacing the largest element 
with a new largest element. The value A(r) is augmented by x — y (it 
already contains 2 — x from Phase 1). The element z replaces zx. 

If k ~7,, we push x down into second place, since z is now the 
largest element, and subtract x — y from v(t), since x no longer 
contributes. 

After this update has been done for each column, we copy A(r) into 
v(k) and interchange rows r and k. (In the actual implementation, of 
course, row movement is just pointer manipulation.) Now go back to 
Phase 1. 

This is the end of the algorithm description. The critical part of this 
algorithm is evaluating the contribution of a row without doing any 
of the updating necessary to exchange it, and particularly without 
scanning the matrix to find any column maxima. This latter operation 
need be performed only after we have decided upon an exchange; 
furthermore, it is performed only upon a small set of columns—those 
for which the element of the row being replaced was first or second 
largest and for which the replacement element is smaller than both 
[case (t) in Phase 3, above ]. In practice, this condition holds for about 
10 percent of the columns when we actually do a replacement. Since 
typically we replace relatively few rows in proportion to the number 
examined, the time saving is large. 


iI. AN EXAMPLE 


This description may be clarified by one step of an example. Suppose 
m = 8, the cost matrix is 


P= 


TR be 
© bo Oth 
mm Co 00 0 
eo NO 
WoOrRNrH 
oOnNnNr Oo 


and the first three rows are the current set. (This is obviously not a 
probability matrix—small integers are better for exposition.) Then the 
best entries are (2, 5, 8, 7, 4, 6), a value of 32. 

Initialization. We find that v1 1s 5 (from column 6), v2 is 7 (from 
columns 1, 3, and 4), and v3 is 4 (from columns 2 and 5). Thus 
v = (5, 7, 4). Suppose we want to evaluate row 4 as a replacement for 
one of these rows. 
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Phase 1. Considering column 1, 4 > 2 so we are doing case (777). The 
value A, is augmented by 2; at the same time, A» is decreased by 1, 
because the element in row 2 is no longer largest in column 1. 

There is no change in column 2 [case (iz) ]. In column 3, row 4 
represents a new second-largest element, so Az is decreased by 1. There 
is no change in column 4. In column 5, add 4 to Au, and subtract 2 from 
A;. In column 6, decrease A; by 1. Thus, A = (—1, —2, —2, 6). 

Phase 2. The minimum of v; + A; is 2, at 7 = 3, implying that row 
3 should be ejected. We now commence the updating operation. 

Phase 8. For column 1, the largest value is 4 (coincidentally in row 
4) and the second largest is 2 (in row 2). Decrease v2 by 1, since row 2 
no longer contributes in this column [case (772) ]. In column 2, we are 
replacing the largest element by a very small one [case (2) ]; the old 
number two element becomes the new largest, and we have to search 
for the new second largest. Add 1 to v4, since the largest value is a 3 
and the second largest a 2. In column 3, p13 represents a new and 
bigger second element; decrease v2 by 1. No change takes place in 
column 4. In column 5, we are replacing the largest element by a new 
largest element. The value of vs is increased by 2(ps,3 — ps,2). In 
column 6, we gain a new second element, so v; is decreased by 1. 

En route, we compute the new value of the solution as 36. 

When we have finished, v = (5, 5, 8) for rows 1, 2, and 4, and the 
process continues by our considering row 5. Notice that out of six 
columns we only had to search for a second-largest element once; the 
rest of the time, it was immediately at hand. 


IV. EXPERIMENTAL RESULTS 


We have tried the procedure on several different types of data: 
matrix entries random on {0, --- , n}, random 0 — I matrices, and 
various probability matrices based on the physical problem. 

For problems formed by generating random entries in the matrix, 
the fraction of random starts producing the optimum diminishes as 
m/€ increases (except for the trivial cases of very small or very large 
m), and also as n increases. Although there is significant individual 
variation, the frequency of obtaining the optimum is close to 100 
percent for small problems and still about 20 percent for problems 
with m = 10, ¢ = 60, n = 80, the largest random problems tried. (It 
should be noted that “optimum” usually means “best solution seen 
in a large number of trials’; we have strong statistical grounds for 
believing them optimal, but no proof.) 
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TaBLE I—Typicau STATISTICS FOR SMALL. RANDOM PROBLEMS - 
(ENTRIES UNIFORM ON [0, 7 ]) 


No. of Lea 6 


: optimums/ “pee 
8 - Time Total Trials polutont 
m £ } n- - (ms) (range) - (range) : 
5 10 20 21 -. 20/20 | 1 
5 40 40 120 32-39/50 6-9 Sa 
5 40 80 230 6-47/50 4-19 ' 
10 20 20 42 20/20 1 
10 40 40 140 25—41/50 3-5 
10 40 80 300 11-16/50 5-10 . 
20 60 60 340 18-41 /50 2-5. 
20 60 80 - 450 1242/50 7-13 


_ Run time is directly proportional to én; in absolute terms, the rtin 
time is about 100 én microseconds per random start. 

The procedure almost always makes less than two passes through 
the £ — m unused elements; the number of row replacements is roughly 
equal to m. As we mentioned above, it is only on these occasions that 
it is necessary to actually update the records of first- and second-best 
elements, and only for about 10 percent of the columns among the — 
replacements is it necessary actually to scan through the m rows. to. 
locate a new second largest. (After initialization, it is never necessary 
to scan to find the largest.) Table I shows some typical results for thése 
smaller tests. 

Limited tests on 0-1 matrices Sroduced similar results, although the 
run time appears to be slightly lower per case. It is possible in the » 
0-1 case to make several simplifications that would further decrease — 
run time, and storage requirements could be drastically reduced by’ 


-0-0:0- 


Fig. eee ee 
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using bit storage. However, we have not experimented extensively on 
- 0-1 matrices. This will be reported in a separate paper. 
_ Several experiments have been performed on various models of the 
‘real problem. Since the modems involved have very low error rates, 
the error transition probability p.;,7  j, is small. This means that the 
matrix P has large diagonal elements, a few small elements, and a 
large number of zeros. For our purposes, the existence of large values 
is irrelevant; however, many zeros means that a storage organization 
that does not store zeros is attractive. We will discuss this shortly. 
One problem to be faced is how to represent the continuous (a, b) 
space as a set of discrete points. One crude model we studied uses a 
“honeycomb” or hexagonal scheme (shown in Fig. 1), since this is a 
reasonable approximation to circular symmetry. The inner 61 points 
represent allowable signal locations; the outer 30 are the extra points 
to take care of the set of received signals that violate the peak power 
constraint. 
_ For this test, P is defined as 


pi; = e ifj is a neighbor of 7 (e < 1) 
1 — (e X number of neighbors) ift=7 
= 0 otherwise. . 


ll 


This set of probabilities ignores phase jitter, which adds a radially 
increasing tangential component to the error transitions. 

Figure 1 shows an optimum solution (it is easy to prove it optimum); 
Fig. 2 shows a local optimum differing by «. In both cases, 12 signals 
_ are placed symmetrically on the boundary, and the remaining four are 
placed as well as possible in the interior. For this problem, all solutions 
were within 9¢ of optimum and the median within 3¢ (the mean ran- 
dom start is about 35¢ away), run times averaged 370 ms per case, and 


LG Os; 


Fig. 2—Suboptimal by one unit. 
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the frequency of optimum solutions was about 10 percent. This problem 
appears to be slightly harder than random problems of this size, but 
run times are smaller. 


V. LARGER PROBLEMS 


To properly handle larger problems (e.g., to increase the resolution 
available when discretizing) a version of the program using a sparse 
matrix representation has been implemented. This involves substantial 
overhead in accessing elements of the matrix, but it is balanced by the 
fact that, when most matrix elements are zero, much less processing is 
required. For example, in the 16/61/91 (i.e., m/£/n) problem above, 
average run time increased from 370 to 390 ms, which is not significant. 
The run time is determined predominantly by the number of non-zero 
points, which is usually proportional to fn. 
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Fig. 3—“‘5-11” solution in pure Gaussian noise; 6 = 0.0; errorrate = 1.07 X 1077. 
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TABLE I]—SomeE RESULTS ON DESIGN PROBLEMS 
(m = 16, No = 0.002) 





Phase 
Run Jitter Character 
Non-zero Time B of Best 
£ n Entries (s) (degrees) Solution Figure 
1. 293 421 8,500 6.5 1.5 1-5-10 - 
2. 421 577 17,000 14.5 1.5 1-6-9 —_ 
3. 421 577 17,000 14.5 1.5 1-5-10 _ 
4.; 489 665 22,000 19.3 0 5-11 3 
5. 489 665 22,500 18.4 1.5 1-5-10 4 
6.| 489 665 27,000 20.5 3.0 ee 5 
peci 
7.| 489 665 27,000 21.0 3.0 desion 6 
8. 577 749 31,000 24.9 1.5 1-—5-10 - 


VI. RESULTS ON REAL PROBLEMS 


In this section we discuss some of the experimental results obtained 
from real problems, using formulas for the transition probabilities 
taken from Reference 2. The probability for the transition from 
X = (21, %2) to Y = (y1, yz) has the general form 


p(X, Y) = f(X, Y; 8, N.) exp Lg(X, Y; 8, No) I, 


where J, is the noise power of the channel and 8 is the rms phase jitter 
in the received signal. The details of f and g do not concern us here; it 
is sufficient to say that p(X, Y) drops to zero rapidly as Y gets further 
from X. For example, in the pure Gaussian noise case (@ = 0), we have 


p(X, Y) = x exp | vor | 


Thus, the probability matrix is quite sparse when the transition 
probabilities have been scaled and converted to integers. 

The discrete space consists of points on a square lattice, as shown in 
Fig. 3. The number of rows in P is determined by the number of lattice 
points within the circle of radius 1. To these are added exterior points 
approximating all possible additional received points. Since the radial 
probabilities drop off rapidly, this exterior layer need only be one or 
two units thick. This extra layer is indicated by the band of omitted 
points on Fig. 3. 

The run time and the storage requirements both grow with the 
number of lattice points; this limits the resolution we can use. The 
largest problem tried had 665 rows, 861 columns, and about 40,000 
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Fig. 4—“1-5-10” solution for 1.5-degree jitter (error rate = 2.97 X 107). 


non-zero matrix entries. It should be noted that the matrix has a 
fourfold symmetry, so, at the price of an increase in computing time 
(in practice, about 50 percent), only 10,000 entires need be stored. 

Two types of experiments were performed. First, several constella- 
tions of intrinsic interest were used as initial solutions; the heuristic 
procedure attempted to improve upon them. Second, the procedure 
was used to produce good solutions from a large number of random 
initial configurations. For all solutions, approximate error rates were 
computed and the constellations displayed. 

Table II lists some typical parameters for several experiments at 
various sizes; Figs. 3 to 5 show the best solutions found for particular 
parameter settings. 

Each signal point is surrounded by a set of points which, when 
received, will decode into that signal point. This set of points is the 
‘decision region’’ for that signal point. Because we have quantized a 
continuous space into small squares, the decision regions surrounding 


SIGNAL DESIGN OPTIMIZATION 1157 


e 
e 
ee @-e. 7 © 
. r) e e ° 
r) ee e . *“@e. 
e e 7. e¢ ¢e @ ee @ 
e e e ee 86° ° 
@- e U6©8t e- ee © ° 
e0Uhl8f e -@- e 
e e >. @ e - @ 
e ° e e e e e 
e e - @ e e 
» - @ e e 
e e e 
e “2 @e 
e ee e @ 
e e e-)0Ue e e.0U6Um.e « e °@ 
e e « e @ [on +27 @-e C 
e @- e.|0Umf ee. ¢ >. e¢ °@ 
e e.)hUm e ee. ¢@ ee. @ 
e en ee ee oe ¢ 
° e “ee @e *. @ 
e . ° ee e« oe @ e 
* C) ° ee °° © © e «© @ ° 
oe ° . e © © © © © © e ee © « & ° 
ee @ ee o e e« © e@ @ « «@ 
o © « « ° @- + «© « ° 
) 


Fig. 5—‘‘1-6-9” solution for 3-degree jitter (error rate = 3.26 X 10-5). 


each signal point have “‘ragged edges” and are of necessity somewhat 
arbitrary. As the resolution is made finer, this effect is less serious, and 
in fact the procedure can make more subtle choices of points and of 
boundaries, so the apparent error rate decreases with increasing 
resolution. For this reason, error rate comparisons between different 
resolutions are not appropriate. However, the rates are internally 
consistent in that, for any given resolution, the solution character and 
error rates vary with noise as would be expected. 

As predicted by analytic techniques,” solutions like ‘‘1-5-10” and 
“1-6-9” are better for high jitter (@ > 1.5°), while ‘5-11’ solutions 
are better in low jitter cases. (The notation will be evident after 
examining the figures.) These trends are clearly indicated in Figs. 3 
through 5, which show, respectively, the best solution (a 5-11) for zero 
phase-jitter (pure Gaussian noise) with an error rate of 1 X 107’, the 
best solution (1—5-10) for 1.5 degrees of phase jitter (error rate 
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Fig. 6—Competing design, for 3-degree jitter (error rate = 3.47 X 1075). 


3 X 10-7), and the best solution (1-6-9) for 3 degrees of jitter (error 
rate 3 X 10-°), 

For comparison, we experimented with the competing design shown 
in Fig. 6, at various levels of jitter. This design is intended to be robust 
over a wide range of jitter. This configuration does degrade less than 
the others as jitter increases, but its overall performance is very much 
inferior. Figure 6 shows that, at 3 degrees, its error rate is 3.5 X 10-5, a 
factor of 10 worse than the 1-6-9 configuration. These results agree 
closely with predictions of independent theoretical studies.? 


VII. CONCLUSIONS 


As a solution to a combinatorial optimization problem, the heuristic 
procedure presented here is quite good for small-to-medium problems, 
say up to about 100 rows, even for dense matrices. It remains useful, 
but not strong, for large sparse problems. 
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As for the original design problem, the number of rows and columns 
in the matrix both rise with the resolution; thus, highly accurate 
representations are computationally expensive, so the procedure is 
generally not appropriate for generating precise answers to specific 
design questions. Rather, it is most useful in providing quick approxi- 
mate and comparative optimizations or evaluations, either to furnish 
insight or to supplement results obtained by analytic techniques. 

The extension of this technique to problems with an average power 
constraint, rather than peak power, appears to be straightforward, 
although we have not implemented it. [The average power constraint 
requires that >> (a? + b?) < 1.] 

As the simplest solution, start with a random feasible set of m rows. 
Then, before each possible replace row is selected, test to see if it would 
violate feasibility; if so, it cannot be used. A more powerful algorithm 
would permit temporary violations of feasibility in a controlled way. 
Either of these approaches should serve reasonably well. 
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Impulse Response of Fibers With 
Ring-Shaped Parabolic Index Distribution 


By D. GLOGE and E. A. J. MARCATILI 
(Manuscript received March 6, 1973) 


The index distribution in the cross section of a multimode fiber has an 
important influence on the modal group velocities and, hence, on the fiber 
impulse response. In this paper we derive a method for the evaluation of 
arbitrary circular symmetric index profiles. In particular, we compute the 
impulse response of a fiber with a ring-shaped parabolic index profile 
which exhibits useful equalizing properties. The pulse spread is found to 
be nearly one order of magnitude smaller than that of a fiber with an equal, 
but abrupt, index decline from core to cladding. 


I. INTRODUCTION 


Multimode operation of optical fibers relaxes the fabrication toler- 
ances, allows the use of incoherent sources, and can alleviate handling 
and splicing problems. Modal (group) delay differences are nearly 
equalized!-? if the core index decreases as the square of the fiber radius 
from a maximum at the axis (Fig. 1). A distribution of this kind is 
realized in the Selfoc* fiber, which was indeed reported to have very 
low values of differential mode delay.*:4 

Since then, the question has been raised whether there are other 
index profiles which have similar equalizing effects, but are otherwise 
perhaps more amenable to certain fabrication techniques or have 
advantages with respect to splicing or bending. Although the latter 
part of this question is difficult to answer at this time, it is certainly 
possible to identify at least one profile that has quite effective equaliz- 
ing properties. Imagine a slab with a square-law index distribution in 
transverse direction. The group velocities of all its modes are known to 
be nearly equal.! It is then plausible to expect that these properties 
are approximately preserved if the slab is warped in a way which 
results in a tube with the cross-sectional index distribution shown in 
& ig alae trademark of Nippon Electric Co., Ltd. and Nippon Sheet Glass 

0., ; 
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n(r) =n (1 —Ar2/a?) 


INDEXn 


No(1 —A) 


RADIUS r 


Fig. 1—Concentric parabolic index profile. 


Tig. 2. Assume that a cladding material of lower refractive index fills 
the bore and surrounds the tube to the outside, so that a fiber is formed 
which guides modes within a tube-like structure with parabolic index 
distribution. 

The purpose of this paper is to identify the modes of this structure, 
calculate their group velocities, and predict the impulse response to 
be expected when all modes propagate uncoupled and with equal 
power. To do this, we employ the WKB description® in a form which 
ignores the anomalies of dielectric waveguide modes near cutoff, as- 
suming that few of all the propagating modes are close to this condition. 
For the sake of simplicity, we also restrict the following computations 









INDEX n 


no(1 —A) 


n= =p ape nn a =o 


RADIUS r 


_ Fig. 2—Cross section through a ring-shaped parabolic index profile. Maximum 
index no along a circle of radius Ro. Cladding index no(1 — A). 
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to small index variations, so that all propagation directions can be 
assumed paraxial to the waveguide axis and the corresponding ap- 
proximations apply. 


II. A CHARACTERISTIC EQUATION FOR CIRCULAR SYMMETRIC INDEX 
DISTRIBUTIONS 


Let us adopt a cylindrical coordinate system (r, ¢, 2) and assume 
that the refractive index n is a function of 7 only. We define a local 
wave number 

k(r) = 2rn(r)/d, (1) 


where \ is the wavelength in free space. Because of the circular sym- 
metry, we can separate the general wave equation and solve for ¢ and 
z. In doing so, we define an axial propagation constant 8 and describe 
the azimuthal periodicity by an azimuthal mode number v. The re- 
maining partial differential equation for the radial field dependence 
E(r) has then the form 


oH 1 oH 
or Yr or 





(220 ~s-4)E = 0. (2) 
Following the usual WIXB approach,°® we substitute 
E(r) = &, (3) 


ignore the second derivative 0?u/dr?, and, by solving for du/dr, we 
obtain the solution 


a =— — 3 oe infkXr) —fP-— ( “t ae (4) 


Given @ and », we can find two radii, Ri and Re, at which the root in 
(4) vanishes (Fig. 3). These radii define a ring-shaped region within 
which eq. (4) has an imaginary part causing the field HZ to be a periodic 
function. Outside of the region, # decreases or increases aperiodically. 

As in the 2-dimensional case,* decreasing (or evanescent) field charac- 
teristics outside are obtained if the total phase inside the region is 








[, POPC FD Mer = utd, ©) 


where yp is an integer called the meridional mode number. It determines 
the number of half periods of FE in radial direction. The accuracy of 
(5) improves for large u, but is in most cases surprisingly good even for 
small values of u. Equation (5) permits an evaluation of the propaga- 
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Fig. 3—Sketch defining regions of periodic and aperiodic field characteristics of a 
mode of azimuthal order »v. 


tion constant 6 for given mode numbers u and v. This will now be done 
for the parabolic ring structure sketched in Fig. 2. 
III. GROUP DELAY AND IMPULSE RESPONSE 


Figure 2 shows a cross-sectional view of the circular symmetric index 
distribution. The index has a maximum value 7 atr = Ro, decreases as 


n(r) = nll — A(r — Ro)?/a?] for R-a<r<R+a, (6) 
and has a constant value 
n(r) = nol — A) (7) 


everywhere else. We assume A to be small compared to unity, introduce 
the abbreviations 

ko = 2rno/d (8) 
and 

p=r— Ro, (9) 
and obtain, with the help of (1) and (6), 


k?(r) &Y kg(l — 2Ap?/a?). (10) 
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In order to solve eq. (5) analytically, we assume in addition that 
Ry > a, which permits us to replace r by Ro in (5). As a result, 


(ut De = [Ve — OFF Y/R = 2ARe/e? dr, (11) 


which has the solution 


B = [kg — 0? + 4)/R§ — 2V2A(u + 3)ho/a]}. (12) 


The phase constant 8 of a propagating mode must furthermore 
fulfill the condition 


B S$ ko(1 — A) (13) 


for the cladding field to have evanescent characteristics. This permits 
us to calculate the total number of propagating modes. Keeping » 
fixed, we first determine the number of modes m in a group with the 
same v. We do this by solving (12) for u» with 8 = k)(1 — A). Since 
A<l 


a(v? + 1/4) i 3 
2koR3V2A 4 


This number decreases as y increases. The largest possible v is obtained 
for m = 1. Thus with (14) 


M = Umax(v) + 1 = VA/2ak, — (14) 


2 i 
Vmax = (218Réa — “ho \573) -7 (15) 
For the following approximations, we ignore the terms }. In this case, 
the sum over all m from vy = 0 to ymax yields 
M = 3ak2RvA, (16) 


which is the total number of propagating modes. Using the same 
approximations, we can express m with the help of (15) in the form 


m = VA/2ako(1 — v?/v2ex), (17) 


an expression which will be used later on. 

To calculate the mode delay, we first convince ourselves with the 
help of (14) and (15) that the u- and »-terms in (12) are small (of the 
order A) compared to kz. We therefore approximate 8 by 


B = ko — (v? + 4)/2koR§ — V2A(u + 4)/a. (18) 
The differentiation of 8 with respect to the radial frequency 
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Fig. 4—Impulse response of the thin parabolic tube structure; (a) individual mode 
groups, (b) power distribution for large mode numbers (Ry > a). 


w= cko/no yields the group delay 


oo = no E ca ( es i) / 2888 | (19) 


where L is the fiber length and c the velocity of light in free space. 
Since ¢ depends only on » but not on uw, mode groups with the same v 
have the same delay. Consequently, if all modes are excited by equal 
pulses of unit energy at the fiber input, the output consists of pulses 
of energy m(v) delayed by ¢(v). If we ignore the delay Lno/c common to 
all modes and then insert (15) into (19), again neglecting the terms 4, 
we can write the delay in the form 


rv) =t- ==. (20) 


Figure 4 illustrates the output distribution for the case in which the 
pulses are so narrow that individual groups are resolved. All pulses 
have the same (very small) width, and their heights correspond to the 
total energy m in each group. 

More meaningful than this plot is a plot of the energy per unit time 


p(r) = mdy/dr (21) 


which coincides with the power distribution in the case of very large 
mode numbers. We find d7/dy by differentiating (20) and, if the term 
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Fig. 5—Impulse response of the parabolic tube structure for various radius-to- 
thickness ratios. 


+ in (14) is again ignored, we have 


_ cakok [ (ALno : cr \3 
pr) = v2Ino | ( Cr ) (az) |: (22) 


This function is also plotted in Fig. 4. We find half the total output 
concentrated in the time interval 


T = 0.12AnoL/c. (23) 


The remaining power is drawn out in a tail of length AnoL/c. This tail 
is caused by modes of high azimuthal order », modes which are not 
present in the 2-dimensional structure and are essentially equalized in 
the case of the concentric parabolic profile (Fig. 1). In this respect, the 
parabolic ring structure is inferior to the corresponding concentric 
profile, yet an effective width of 0.12 AnoL/c may be a useful improve- 
ment in comparison to a guiding structure with uniform index n» which 
theoretically produces a width AnoL/c. 

The condition Ry > a was necessary for an analytic solution of the 
integral (11). Exact numerical results for arbitrary ratios Ro/a are 
shown in Fig. 5. The corrections with respect to (22) are largest for 
high azimuthal orders. An exact analytical solution can only be found 
for the maximum delay r(ymax) which becomes 


T(¥max) = Atel E _ ; (\@+s = =) |. (24) 
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As an example, consider a parabolic ring whose half width, a, is 
equal to the central radius Ry. Let no = 1.5 and assume A = 1 percent. 
The total width of the impulse response after 1 km of this fiber would 
be r(vmax) = 25 ns according to (22), but half the power is concentrated 
within the first 6 ns. Since high-order modes are usually lossier than 
the low orders, it is likely that much of the pulse tail does not reach 
the fiber end. 


IV. CONCLUSIONS 


The WKB approximation yields a simple characteristic equation for 
the propagating modes in fibers with arbitrary circular symmetric index 
distribution. We use this method to compute the impulse response of a 
fiber with a ring-shaped parabolic index distribution. We find that this 
structure has equalizing properties similar to the concentric parabolic 
index distribution, except for certain azimuthal mode orders, which 
lag behind, forming a rather long pulse tail. The rest of the power is 
concentrated in a time interval which, for a 1-km length and a relative 
index difference of 1 percent, is only 6 ns. 
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The Impulse Response of an Optical 
Fiber With Parabolic Index Profile 


By D. MARCUSE 
(Manuscript received March 6, 1973) 


To the paraxial approximation there is no difference in the group delay 
of the modes of a parabolic index fiber. However, the wave optics treatment 
of the infinitely extended parabolic index medium predicts a slight differ- 
ence in the group delay of the various modes. This result is used in this 
paper to predict the shape and width of the impulse response function of 
a parabolic index fiber with finite radius. 


I. INTRODUCTION 


The current interest in multimode optical waveguides is related to 
progress in the fabrication of luminescent diodes which have become 
cheap and dependable sources of incoherent light. Since incoherent 
light cannot be injected into a single-mode fiber with high efficiency, 
multimode waveguides must be used. A disadvantage of using multi- 
mode instead of single-mode waveguides is multimode pulse dispersion 
caused by the fact that the group velocity of the guided modes is not 
the same. Power injected at one end of the waveguide is shared by 
many or all of the possible guided modes. As each mode reaches the 
other end of the guide at a different time, the initial pulse is broadened. 

It is the purpose of this paper to calculate the impulse response of a 
graded-index, multimode fiber.1~* The index distribution is assumed to 
be given by the expression 


llA 


r2 

n=n,(1~a5) Osrsa. (1) 
The parameter a represents the finite radius of the fiber, A determines 
the strength of the index gradient. The analysis is simplified by using 
the modes of the infinitely extended square-law medium (1) instead of 
the modes of the actual waveguide of radius a. 

The impulse response of a graded-index fiber with parabolic index 
profile is much more favorable than that of the usual clad fiber with a 
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rectangular index profile. In the paraxial approximation it is a delta 
function. The finite width of the impulse response function is attribut- 
able to rays that move on trajectories making relatively large angles 
with the waveguide axis. 


II. THE MODES OF THE SQUARE-LAW MEDIUM 


The electric or magnetic field components of the modes of the 
infinitely extended square-law medium are obtained from the reduced 
wave equation 





2 
Sa t Gt vik’ — BY = 0, (2) 
with 
m= ng(1—- 20 EE). (3) 


The coordinates x and y are oriented in transverse direction to the 
waveguide axis which points in z direction; k = 27/) is the propagation 
constant of light in vacuum and @ is the propagation constant of the 
guided modes. The distribution of the square of the refractive index is 
not simply the square of the index distribution (1). Equation (3) 
follows from (1) only if we neglect the square of the A term. However, 
we can turn the argument around and consider eq. (8) as correct and 
(1) as the approximation. The exact solution of (2) with the function 
(3) is given by* 


vu = AH, (2=) Hy (24) ep (= a) cies, (4) 


H, is the Hermite polynomial of order p. The beam half-width is 


defined as 
2\t/a \ 
(ee (Z ) ( Nok ) 5) 


and the propagation constant is given by 


p=nk [1-24 o+0+n] 








(6) 


However, eqs. (4) through (6) are not an exact solution of Maxwell’s 
equations since an additional term containing the gradient of n? has 
been neglected in (2). Since this additional term does not make a 
significant contribution in (6)—particularly at large mode numbers p 
and g—we use the solution in its present form for our discussion of the 
impulse response of the fiber with parabolic index profile.® 


PARABOLIC INDEX FIBER 1171 


III. GROUP DELAY 


We are interested in fibers satisfying the inequality 


VA 


n Ka 





«1. (7) 


We thus approximate (6) in the form 


V2A A 
B= nk —-——@tatD—- Faw tat i)’. (8) 
The group delay can now be expressed as 
~L_L468 _ nb A 
"oy edk ¢ [++ Gackaye @ +a + 0"): . 


v is the group velocity and L is the length of the waveguide. The second 
term of 8 in (8) does not contribute to the group delay. In first approxi- 
mation, if the third term in (8) is neglected, the group delay would be 
independent of the mode number. The difference in the group delay of 
the different modes is thus only slight in the square-law medium. 


IV. CUTOFF CONDITION 


In the infinite square-law medium there is an infinite number of 
modes; p and qg can both assume values from 1 through infinity. The 
number of guided modes of a fiber with radius a must be finite. It seems 
reasonable to assume that those modes that interact strongly with the 
waveguide boundary at r = a lose power at a high rate and become 
unimportant for the power transport. Low-order modes are concen- 
trated near the waveguide axis while the modes spread out further 
away from the axis with increasing mode number. At a certain mode 
number the modes reach into the region of the fiber boundary and thus 
become very lossy. 

It is known from the theory of the WKB approximation® that the 
mode field has an oscillatory behavior in the range 


n(r)k > B (10) 
and an exponentially decaying behavior in the range 
n(r)k < B. (11) 


It appears logical to let the cutoff point of the guided modes in the 
fiber with parabolic index profile and finite radius r = a coincide with 
the condition 

n(a)k = B. (12) 


Using the square of (12) and eqs. (3) and (6) results in the cutoff 
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condition 
A 
S=(»+9).= a ncka. (13) 
Using (5) we can also write the cutoff condition in the form 
ga 14 


The actual performance of the fiber with parabolic index distribution 
can now be approximated by assuming that all modes carry equal 
amounts of power up to the maximum mode number that is determined 
by (18) or (14). 


V. THE IMPULSE RESPONSES 


The impulse response of the fiber with parabolic index profile is 
obtained by counting the number of modes that arrive at the fiber out- 
put simultaneously. The power carried by these modes is proportional 
to their number. The waveguide losses do not influence the shape of 
the impulse response function if we assume that all modes suffer equal 
amounts of loss. 

Equation (9) shows that modes with constant values of 


u=pt+q (15) 
arrive simultaneously at the end of the waveguide. The number of 
modes with equal group delay is obtained by inspection of Fig. 1. The 
modes of the waveguide occupy the area of the triangle indicated in the 
figure. Modes with equal transit time lie on the straight line labeled 
uw = const. The area in 7p, g space is equal to the number of modes con- 
tained in it. The number of modes in the interval dh is thus given by 
the length of the line uw = const times dh. The length of the lines 
u = const is 2h. We thus have for the number of modes in the in- 
terval dh 


M(h)dh = 2hdh. (16) 


The total number of modes is 
(1/8) 8 
ee / M(hdh = 38°. (17) 
0 


The ratio of M(h)dh/N is equal to the ratio of the power AP, that 
corresponds to the interval dh, divided by the total power P arriving 
at the end of the waveguide. We thus have 


1dP _ 4h 


P dh ~ S® oe 
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uU S p 


Fig. 1—Mode distribution in p-g space. The dotted line labeled u = const corre- 
sponds to modes with equal group delay. 


We define the impulse response function F(7) as the relative amount of 
power arriving per unit delay time 7, 


MO) = B dp > P dh dr ue) 
All that is left to do is the determination of the function h(r). We see 
from Fig. 1 that 
1 1 








h=zu=Z@ta). (20) 
Neglecting the 1 compared to p + q in (9) we have 
noka [ cr ); 
h= —1). 21 
V2A (a7 oe 


The impulse response function is now obtained by combining eqs. (18) 
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and (18) through (21). 





CT 
0 Pe 
2c c 
F(r) am noLA? 0 ier 1 < 4A?, (22) 
CT 
0 we 1 > 34” 


VI. DISCUSSION 


Equation (22) shows that an impulse, shared equally by all the modes 
at the beginning of the parabolic index fiber, reaches the end as a 
rectangularly shaped pulse whose width is 


Nol 
nae 


The pulse width is thus dr = 0.25 ns/km for n, = 1.5 and A = 0.01. 
The pulse width increases rapidly with increasing values of A. For 
A = 0.015 we have a pulse width of dr = 0.55 ns/km. However, the 
impulse response of the parabolic index fiber is much more favorable 
than the corresponding impulse response of the conventional fiber with 
discontinuous index distribution whose impulse response width is 
directly proportional to’ ni/n2 — 1 (m1 = core index, ne = cladding 
index). 


AP. (23) 
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Baseband Linearity and Equalization in 
Fiber Optic Digital Communication 
Systems 


By 8S. D. PERSONICK 
(Manuscript received February 21, 1973) 


If a sequence of digitally on-off modulated optical pulses is injected 
into a dielectric waveguide, these pulses may begin to overlap after a 
sufficient distance of propagation because of material dispersion and/or 
group delay spreading. In general, the pulses will not add linearly in 
power, which can complicate the problem of equalization of the square-law 
(power) detected overlapping output pulses at baseband. This paper 
illustrates «important situations in which the guide may be treated as 
‘“nseudo-linear”’ in power, meaning that the detected guide output pulses 
appear to add linearly. 


I. INTRODUCTION 


If a single pulse of optical energy propagates along a dielectric 
waveguide, pulse broadening can occur for one or more of the following 
reasons: material dispersion, individual mode waveguide dispersion, or 
differences in the group delays of different guide modes. In addition, 
the pulse shape may become only statistically defined because of 
random mode coupling and/or statistical fluctuations of the optical 
source. 

If a sequence of digitally on-off modulated pulses is injected into a 
dielectric waveguide, those pulses may begin to overlap after a sufficient 
distance of propagation. In general, the optical powers in the pulses 
will not add in a linear manner.t On the other hand, as will be shown 
below, the guide may be pseudo-linear in power. That is, for the 
purpose of processing the power received at the output end of the 


t The fiber is a medium which is linear in E field propagation (from Maxwell’s 
equations). It is usually excited by a power-modulated source, and its output field is 
detected by a square-law (power sensitive) device. Even if the input pulses are 
separate, the response of the square-law detector will in general contain cross terms 
resulting from the overlap in the output pulses. 
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guide—in order to make decisions as to whether or not each pulse is on 
or off—we may be able to treat the individual overlapping output 
pulses as if they added linearly. 

If the output power pulses could be considered to add linearly, then, 
after detection, the resulting current pulses could be separated with a 
linear equalizer provided the shapes of the pulses are well defined and 
identical from pulse to pulse. In this paper we consider a number of 
interesting cases in which the guide can be treated as if it were linear 
in power (output pulses add linearly) and in which the pulses at the 
output assume a well-defined shape in spite of random mode coupling 
and/or source fluctuations. 


II. CASE I: MULTIMODE GUIDE, MODE-LOCKED SOURCE, NO MODE 
COUPLING 


The easiest case to visualize in which the guide appears linear in 
power with overlapping output pulses is that of a multimode guide 
propagating pulses derived from a mode-locked laser operating in a 
single spatial mode. It is assumed that there is no mode conversion 
and that group delay differences among the modes dominate pulse 
spreading. 

Even though the laser puts out a well-defined spatial mode, it may 
be very difficult to match this to a given fiber mode so that only one 
fiber mode is excited. We assume that a number of fiber modes are 
excited by the pulses from the mode-locked laser. The assumption of a 
mode-locked laser implies that the optical bandwidth being used is 
small so that material and waveguide dispersion can be neglected, and, 
in addition, no random fluctuations are present from beating of un- 
locked source modes. The sequence of nonoverlapping pulses from the 
laser exciting the guide input will produce identical sequences of non- 
overlapping pulses in each mode at the guide output. However, the 
sequences at the output in the various modes will have relative time 
delays because of the differences in the group delay per unit length 
associated with the various modes (see Fig. 1). When the fiber output 
falls upon a detector, the current produced (neglecting shot noise) is 
proportional to the sum of the powers in all the modes. The sum of the 
powers in all the modes resulting from a single input pulse is shown in 
Fig. 1. Since the output pulses in a single mode do not overlap and 
since the detector linearly adds the powers of the various modes, the 
total detected current will be a sum of pulses modulated on and off, 
each of which looks like the response to a single input pulse. Thus, the 
output power produces a detected current which is a filtered version of 
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Fig. 1—Multimode propagation without coupling. 


the input power. That is, the sequence of nonoverlapping input pulses 
produces a detected current at the output, which is a sequence of over- 
lapping pulses that add linearly. For the purposes of processing this 
current, the guide can therefore be considered linear in power or linear 
at baseband. 


III. CASE II: INCOHERENT SOURCE, MULTIMODE GUIDE, NO MODE 
COUPLING 


In this example, we assume that a pulse modulated incoherent 
source excites one or more modes of a multimode waveguide with no 
mode coupling. We show that the received output power in each mode 
can be treated as a linearly filtered version of the input power, i.e., that 
the sequence of nonoverlapping input pulses produces a sequence of 
output pulses in each mode which add up linearly, whether they overlap 
(because of material dispersion) or not. Since the detector produces a 
current which is proportional to the sum of the powers in each mode, 
the total current will also consist of a sequence of pulses which add 
linearly, whether they overlap or not. We show that, in order for this 
effective linearity in power to be valid, it is necessary that the source 
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bandwidth be sufficiently large compared to the reciprocal of the input 
pulse duration. How big the ratio of these two quantities must be de- 
pends upon how much overlap there is in the output pulses and there- 
fore upon how much equalization is required to separate the pulses at 
baseband. 

We can model the complex amplitude of a given spatial mode at the 


guide input as follows: 
ein(t) = Vm(t)c(t). (1) 


In the above example, (m(t))? represents the modulation and c(é) is 
a complex Gaussian random process which represents the incoherent 
carrier. By definition of an incoherent carrier, we have 


{c(t)c(t + u)) = 0, (c(é)c*(t + u)) = R.(u). (2) 


The Fourier transform of R,(u) is what is called the incoherent source 
spectrum, shifted to baseband. 

The input complex amplitude of (1) produces an output from 
the guide in the corresponding mode having the following complex 
amplitude 


enue ‘ cin(t/)hg(t — t/)dt’. (3) 


In eq. (8), h,(u) is the guide bandpass impulse response for the mode 
under consideration. Equation (3) follows from the fact that the guide 
is linear in voltage. 

The average power at the guide input in the given mode is (averaging 
over the fluctuations in the incoherent carrier) 


(pin(t)) = (€in(t) in (2) ) v= m(t) (0) _ RO)LX Arh; a(t = kT’) ]. (4) 


In (4), the modulation m(¢) is a sequence of nonoverlapping pulses 
(modulated on or off) where hp,,(t) is the pulse shape, a; assumes the 
value zero or one for each k, and T is the pulse spacing. The average 
power at the guide output is 


(pous(t)) = ( [ f ccltyhglt = tee ynet - avar"). (5) 


It is reasonable to assume that the following approximation holds: 


(ein(t’) ém(t!”)) = (Win tet’) Vm E)c*(0")) = Vt) Vint") 
: RA’ —t) Se me)R’ — tv’). (6) 


This approximation is valid since the coherence time of a typical 
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incoherent source such as a GaAs LED is of the order of 10—!* seconds, 
while modulation pulse widths of interest here exceed 10~° seconds. 
Substituting (6) into (5) we obtain 


(Dout(t)) = LE Aehpourlt — kT), (7) 
where 


| if hoin(t’) / R(t) — t)hglt — t')hgt — "ae" dt’. 


Thus, the average output power is a linearly filtered version of the 
average input power. That is, the average output power consists of a 
sequence of pulses which add linearly even if they overlap. 

Thus far, we have considered the average output power, averaging 
over the fluctuations in the incoherent source power output. We next 
consider the effect of those fluctuations on the equalized-detected 
current. We shall show that these source fluctuations will produce 
negligible deviations in the equalized-detected current from its mean 
provided that the source bandwidth is sufficiently large. 

We can write the power at the guide output in the mode under 
consideration as the sum of the average power of eq. (7) and a deviation 
from this average b(t): 


Pout(t) = (Pout (t)) oF b(t). (8) 
If this power falls upon a detector, it produces a current which is 
proportional to Pour(é) (neglecting shot noise). This current will pass 


through a filter which performs an equalization function and/or band- 
limiting. The baseband filter output voltage will therefore be 


vouelt) = 2f Pow(l halt — Ud’ (9) 


where ha(t — t’) is the baseband filter impulse response and z is an 
arbitrary proportionality constant. 
The mean baseband output voltage is given by 


(vous(l)) = 2f (Boule) Yealt — ede’ (10) 
The mean square deviation of the baseband voltage from its mean 
is given by 
(s2ae()) = Coal) = 030) © 22 f Dousll)Pour(e”)) 
X Aoi — Uho(t — t’)dt’dt” — Cvourld)) J? (11) 


Thus, to calculate the ratio of the mean voltage to the rms deviation 
in order to determine whether or not the deviations are negligible, we 
need the correlation function of the power Dou:(é). 
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In order to calculate this correlation function, we must recall that 
c(t) defined in (1) is a complex Gaussian random process and satisfies 
[in addition to (2) ] the following 


(c(te* (tet) c*t'")) 
= Rt — t)R.(" — ¢”") + Bt — RA’ — ). (12a) 


Using (1), (8), (4), and (12a) we obtain 
(Pout (t) Pout (?’) ) = (Dout (2) ){Pour(t’) ) 
+ | [m(a)Rela— Bhy(t — a)hg(e! — @)dads|". (12h) 


To obtain some numerical results, we assume that the input pulses 
hy,,(t) defined in (4) are Gaussian in shape and that the guide mode 
impulse response corresponds to that of a dispersive medium having a 
group delay 7» at the optical source center frequency and a dispersion 
y? within the optical band of the source. Further, we shall assume that 
the source spectrum is Gaussian in shape. That is, we shall assume the 
following: 


hot) = exp (—t?/207), 
(2.360 = input pulse width between 4 points) (13a) 
F{ho(t)} & Ha(w) 


= exp (-i) exp (— jrow) exp (—w*Bz?/2) (18b) 
1 ‘ ts = 
ho(t) = VeGy? BS exp [—(t — 7.)?/2(jy? + B;*) ] (130) 
R.(u) = exp (—uw?B%/2), 
where 


¥{ } = Fourier transform. 


In eq. (18), we have already assumed that the source bandwidth B, 
is much greater than the reciprocal of the input pulse width c. We also 
shall assume that the guide bandwidth B, is much larger than the 
source bandwidth B,. Since 7, represents an absolute propagation 
delay from input to output, we shall neglect it as irrelevant to the 
problem at hand. Therefore, the only significant parameter in the guide 
impulse response h,(t) is the dispersion y?. 

If we insert the particular functions of (13) into (7) and (12b) using 
(4), we obtain 


(Pout(t)) = 22 2) ax exp {— 3 — kT)?/[y‘BS + o7]} (14a) 


LINEARITY AND EQUALIZATION 1181 


(i.e., a sequence of Gaussian-shaped on-off modulated pulses having 
width vy4B?2 + o?), where 2. = an arbitrary proportionality constant 
which we shall henceforth set to unity. 


(Pous(t) Poult’) = C{Ppoue(t)) ; + exp | > [FT| ee 
ere 





Thus, /»y.u.(t) of (8) is in this case the Gaussian-shaped pulse in (14a). 
Looking at eqs. (18a) and (14a), we see that y4B2/c? < 1 implies that 
little pulse broadening has occurred in propagation; y‘4B?/c?> 1 
implies that considerable pulse broadening has occurred in propagation. 
Now using (14) in (11), we obtain 


actos (_f (ons(t) Yhalt — thaw’) 


2(i) ag eg. SD) 
= j [pout (t’) halt — t’)hs(t — ten -e aware 


where 
uw? = 2BiLy*B3/o? + 1}. 


Recall that (v.u:(t)) is the average baseband (detected and equalized) 
voltage produced by the power output in the mode under consideration, 
and that it consists of asum of on-off modulated pulses given in (14a). 
In addition, o2(¢) is the mean squared deviation of this baseband 
voltage from its mean. Thus, (15) is effectively a signal-to-noise ratio. 
For a typical broadband source and an equalizer having a bandwidth 
comparable to 1/(pulse spacing-T), we can treat the Gaussian term of 
the integrand in the denominator of (15) as a deltafunction having 
area V27/u. Then (15) becomes 


Court)? _ a i; (Dour(t’) hot — t’) wf 


| (16) 
73(6) wer J (Poult!) 2256 = tae | 


We can evaluate (16) for particular equalizers, hs(t), and particular 
output power pulse widths (7y*B2 + o?). We recognize from (16) in 
general that the equivalent noise b(t) of (8) which must be added to 
the average output power is a signal-dependent noise with correlation 
function 


(6@b@)) = as [(Pour(t)) Pot — ¢), (17) 
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where 
wu? = 2Bi/Ly*B3/o? + 1). 


If the power output pulses hy,,,(f) overlap significantly, then this 
signal-dependent noise will become stationary when all the pulses are 
on, which should simplify the calculation of (16). At the other extreme, 
if the output pulses hp,,,(¢) do not overlap (i.e., y4B? + o? « T*) andif 
the baseband equalizer hy(t) is taken to be a matched filter (matched 
in shape to the output pulses), then the signal-to-noise ratio is given by 


2 
ane = V20B, (18) 
Lif T? >> y*BS + 07, hot) = hut). 

In general, for a given desired equalized pulse shape, the equivalent 
noise will be negligible if the product of the optical source bandwidth 
B, and the input pulse width o is sufficiently large. For practical cases 
of interest, this product is on the order of 104 to 105. For reasonable 
amounts of equalization consistent with other noise considerations 
(shot noise, thermal noise, etc.), we can treat the guide as being linear 
in power even if the pulses overlap, i.e., we can neglect the equivalent 
noise b(t). 

When more than one mode is present, we simply add the individual 
mode output powers, since the detector current is proportional to the 
sum of the powers in all the modes. If the optical source is spatially 
incoherent, the equivalent noises b(¢) in each mode may be uncorre- 
lated.t In that case, the requirements upon the product of the source 
bandwidth and the input pulse width are less stringent. This is par- 
ticularly true if the pulse spreading resulting from dispersion dominates 
the spreading resulting from the differences in group delay among the 
various modes. 

A simple interpretation which may prove useful follows. It was 
easy to obtain (7), which showed that the guide was linear in power 
if we averaged out the source fluctuations. Since the source is very 
broadband, we can think of it as a sum of independently fluctuating 
sources separated by a frequency spacing equal to the bandwidth of 
the modulation pulses at the input. Thus, the output power is the sum 
of the fluctuating powers associated with each equivalent indepen- 
dently fluctuating optical source. The average output powers associated 
with these equivalent sources add systematically, while the indepen- 

t That is, if the optical source is close to the fiber, each fiber mode effectively sees 


an independent carrier, and therefore we obtain averaging of the fluctuations in 
these carriers when the mode powers at the output of the guide are added. 
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dent fluctuations about the average add at random. Thus, for a 
sufficiently large number of equivalent sources—corresponding to a 
large product of optical source bandwidth and input pulse width—the 
total fluctuations become small compared to the average power. In 
effect, one has a frequency diversity system. 


Iv. CASE III: MODE-LOCKED COHERENT SOURCE, MULTIMODE GUIDE, 
MODE COUPLING 


In this example, we consider a spatially coherent mode-locked laser 
source and a multimode guide. Unlike example I, we assume con- 
siderable mode coupling. Once again, the rationale of using a multimode 
guide with a single mode source may be the inability to stably match 
the source to a single-mode guide. Before proceeding, we must model 
the transmission properties of a multimode guide with random coupling. 

Very little is known about the complete statistical properties of the 
guide under consideration here. Any particular guide, which is linear in 
voltage (field), can be characterized as having a set of modes associated 
with an ideal guide having no geometry perturbations (which are the 
source of coupling). The input and output complex envelopes of the 
corresponding input and output optical fields can be expanded using 
the orthogonal guided modes, which together with the continuum of 
radiating modes form a complete orthonormal series. We can relate the 
complex amplitudes in each input and output mode by a matrix im- 
pulse response. That is, calling the complex amplitude in mode k at 
the input e;,,(¢), and the complex amplitude in mode 7 at the output 
Ejour(t), We have 


cio!) =f ena Vise — eae, (19) 
k 


where h,;;,(¢ — t’) is the bandpass impulse response from input mode k 
to output mode j. 

In order to proceed in the analysis to follow, we need at least the 
fourth-order joint statistics of the random processes h,;;,(¢). (The reader 
is cautioned that the term random process refers to the fact that the 
actual h;,(¢) for each 7 and k will be different for different guides be- 
cause of the random mode coupling. For any particular guide which 
does not change its physical parameters in time, h;,(t) is a fixed but 
a priort unpredictable function of time. All averaging and references to 
statistical properties refer to ensembles of guides whose gross physical 
properties are alike.) 

As mentioned above, little is known about the statistics of the 
h;.(t). Rowe and Young,! Personick,? and Marcuse* have shown in 
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various analyses that, for a particular 7 and k, the Fourier transform 
of hjx(t), Hj.(w) can be considered a stationary random process under 
various restricted conditions which include the assumption that the 
optical bandwidth being used is not too large. That is, one may argue 
under restricted conditions that 


(Hjx(w) Hw + o)) = Sjx(o) (20) 


(where the averaging is over an ensemble of guides having identical 
gross properties). 

In another analysis, Marcuse* has shown that, for a particular 7 and 
k and for a sufficiently long guide (so that enough mode coupling has 
taken place), one has 


(| H3x(w) (4) ~ 2(| Hx () oy 


This last result is consistent with (but certainly not a sufficient 
condition for) the possibility that H;,(w) is a complex Gaussian random 
process. 

Based on this admittedly scanty evidence which should certainly be 
explored in more depth, we shall asswme that the Fourier transforms 
Hj,(w) of the h;x(t) satisfy the following conditions which would be 
satisfied if the H,;,(w) were joint complex Gaussian random processes 


(Hjx(@)Him(w + ¢)) = 0, — (Hin(w)Him(w + ¢)) = Syrim(o) 
(Hjx() Hin(w F 0) A nolw ale 0’) Hy q(w ae o’’)) (21) 
as Sjeim(o) Snopg(a” = a’) = Syepqo”)Snoim(o = a”). 


It is hoped that the results which we shall next derive will be qualita- 
tively valid for actual multimode guides with random mode coupling. 

The guide input optical field complex amplitude is given by 
ein(t) = ep(e)er(), (22) 


where e,(p) represents the spatial variation of the field over the guide 
input plane and e,(¢) represents the time variation of the field and 
includes the modulation. We shall assume that the modulation con- 
sists of a sum of nonoverlapping on-off modulated pulses with spacing T 


er(t) = > arhinlé — kT), 


2 
a= 0 or 1 foreachk. 7) 
We can expand the input field in the guide modes as follows: 


€p(p) = > exbz(p) + unguided remainder, (24) 
where ¢;(p) is guided mode k. 


LINEARITY AND EQUALIZATION 1185 


From (20) and (24), we obtain the complex amplitude in mode 7 at 
the guide output in terms of the input complex envelope and an im- 
pulse response 


Ejoue(t) = y fcvec@rhial — v’)dt’ 


= [Onset — t)dt’, where h,(t) 8 > exhj.(t). (25) 


k 


Since h,(t) is a weighted sum of individual responses h;,(t), it follows 
from (21) and the linearity properties of the Fourier transform that the 
transform H,;(w) of h;(t) must also satisfy 


(Hj(w)Hiw + ¢)) = Sie),  (Hi(w) Hj + «)) = 0 
(H;(w) Hj(w + 0) H;(o + 0) Hj(w + 0°’) (26) 
= Sj(a)Sj(o"" — 0’) + Si(0")Sj(o — 0°). 


We next show that the guide may be considered under restricted 
circumstances to be linear in power with a well-defined output pulse 
shape even though the output pulses overlap and even though there is 
unpredictable mode coupling. 

First, we can write down the power at the guide input and at the 
guide output in mode 7: 


pin(t) = | f leo(e)|*a3] X ax |dan(t — RT) |* (27) 
(since hin(t)hi,(t — kT) = 0 for k ¥ 0, and a, = 0 or 1). 
Piow(t) = / e(te(t’)hjlt — vat — t’)dt’dt”. 


Next, we can ensemble average 7,;,,,(t) over the ensemble of similar 
guides to find the average power response. 
In Reference 2, it is shown that (26) implies that 


(hilt — t) hit — t")) = 3(¢ — *)ae — €”), (28) 


where s,(t) = §-!{S;(w)} (inverse Fourier transform). 
Using (28) and (27) we obtain 


(Diow(t)) ai a Arh jou(t = kT), 
where 


hint) = f [in(t) Pay(t — ed (29) 


We thus see that the input power consists of a sum of on-off modu- 
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lated overlapping pulses, and the average output power (averaged over 
an ensemble of guides with identical gross physical properties) consists 
of a similar sum of on-off modulated pulses which in general may 
overlap. Thus, in this ensemble-averaged sense, the guide is linear in 
power with impulse response s;(¢). We must next investigate how an 
individual guide in the ensemble can deviate from this average and 
under what conditions these deviations can be neglected for com- 
munications purposes. 

In order to study these deviations we must consider them in the 
context of a detector followed by an equalizing (or simply band- 
limiting) filter. Since the detector produces a current which is pro- 
portional to the linear sum of the powers in each guide mode at the 
output (neglecting shot noise), we consider the response to one mode 
only for the moment. The voltage at the equalizer output is related to 
the output power in mode 7 as follows: 


vour(t) = 2 J Dirt! hace, si.(t — tat’, (30) 


where /aet.-tiit.(4) is the detector-filter impulse response and z is an 
arbitrary constant. 

The average (over an ensemble of guides) voltage produced is a sum 
of on-off modulated pulses, since we have already shown that the 
average output power in mode 7 is a sum of on-off modulated pulses 


(Wout (t) ) r= Ss AM veut a kT), (31) 
where 


Aoou(t) > [Pinal Mhacs.-t.( ss t’) dt’. 


The mean squared deviation from this average voltage is given as 
follows: 


o2(t) & W(t) — Wour(t))? 
= 2? / (Dio (t’) Diou(t’”) aet.- tit. (2 — #’) | 
X hact.- tire. (t 77, t’’) dt'dt"’ ey (Wout (#) )?. (32) 


To calculate the mean squared deviation, we need the correlation 
function of the output power in mode j. From (27) we obtain 


(Piau(a)Pioa()) = fff [Le @ rae eeett”)] 
x (hila ~ t/)hi(a — th(B — t”)h(B — t”))dé'de’'dt!”"dt!"”". (83) 
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In order to obtain numerical results, we must make some assump- _ 
tions to facilitate the products and convolutions of (33). We assume 
that the guide power impulse response s,(t) for mode j is Gaussian in 
shape. It has been shown! that this should be the case under re- 
stricted conditions for long guides. We assume that the input pulses, 
hin(t), are Gaussian in shape with a width less than 3 the pulse spacing, 
T, so that the previous assumption that they do not overlap is not 
violated for practical purposes. That is, we assume 


hind) = e7P/2%, 30 < T = pulse spacing 
8;(f) = e~#/27*, y > 3c. 


From (83), (26), and (34) we obtain 


(34) 


(Diou(a)) = 2D) a, exp — {( = ety /2| 5 aa ri} 
(PD jour(@) Pioue(B) ) = E my ApAp—mAiAi+m (35) 


a hep 25 } > {573 + 5 \ 
xexp— | eB omer | 
+ (Djou(@) {Piou(B))« 


The approximations of (35) become equalities when the width of the 
average power impulse response y becomes large compared to the input 
pulse width, oc, i.e., when there is a lot of pulse spreading in propagation. 
We shall soon see that this will be the case of interest in this example. 
Before attempting to use (35) in (32) to evaluate the magnitude of the 
deviations of a particular guides power from the ensemble average, we 
can make some simplifications and comments upon (85). Using the 
assumptions y > o, we have 


(Diow(a)) = 2X az exp {— (ae — kT)?/2y?} (36) 
Rola, B) © (Diow(@)Diour(B)) — (Pioule))(Diow(B)) 
22 TIDY Aap mA 14. mel (AFL? 1277 Jeg l— (BIT)? [277] 

kml 


X Ee (a-B-mP)? / 207 | 


II 


If we assume that c is small compared to the pulse spacing T and if 
we assume that the detector-equalizer combination passes frequencies 
only up to the inverse of the pulse spacing 7’, then we can make the 
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further approximation 
Ria, 8B) = 2 Urd ApQk—mA1Qi+m 
X [e(a- bP)? 121° e— (B17)? 1297 15 (q — B — mT)V2r0%. (37) 


What we have shown so far is that the power in mode j at the guide 
output can be considered to be of the form 


Diouwlt) = ((Djou(t)) + b(8), (38) 


where 
(b(a)b(B)) = Ra(a, 8), 


where b(t) represents the deviations of the power in a particular guide 
in mode j from the ensemble average. 

Combining (36) and (38) with (32), we obtain the ratio of the 
(average voltage)? at the detector filter output to the mean square 
deviation from this average voltage 


Sa (our(t))? 
N a(t) 


. 2 
| fz ape HRT Yh ae tn. (E — eat 
(V2) [pps Op.O1A pty me (YET) [242g [= (0-17) 12921 
Xx 5(t’ —{’— mT ’)haet.-tiit.(t _ t’) haet.- tit. (t _ t’’) dt’dt"’. (39) 


It is clear that, whatever the equalizing filter is, this ratio increases 
with decreasing o. Thus, as the input power pulses to the guide become 
narrow, two things happen: the average output pulse widths become 
independent of the input pulse width and the deviations of the output 
power in mode 7 from the ensemble average become negligible. Exactly 
how small o has to be depends upon how much the average output 
power pulses overlap and how much equalization we are therefore 
using. In the extreme case of no output pulse overlap, assuming that 
the equalizer response /act.-rit.() is matched to the average pulse 
power, we obtain 

T > 3y (no pulse overlap at output) 
= sae for o < y (output pulse width > input pulse width) (40) 


haet.-titt.(£) = h;,.(—é) (matched filter equalizer). 


8 
N 


Obviously, the guide acts linearly in power if the output pulses 
don’t overlap, but (40) shows that the output pulses take on a well- 
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defined shape (i.e., the deviations from the ensemble average are 
negligible) in spite of the random mode coupling. For cases where the 
output pulses overlap, the conditions for the deviations from the 
ensemble average to be negligible are more stringent, i.e., the signal-to- 
noise ratio (39) is less than the special case (40). 

Summarizing, we started with a mode-locked laser putting out 
pulses which were much narrower than the spacing between them and 
on-off modulated. This optical field excited a guide with random mode 
coupling. We modeled the transfer function relating the input field 
complex envelope to the complex amplitude of a particular mode at the 
output as having specific properties (26) associated with a complex 
Gaussian random process. This model was justified only in the sense 
that it was consistent with available but scanty analytical results on 
guides with random coupling. We showed that the ensemble average 
output power in the mode under consideration looked like a linearly 
filtered version of the input power. That is, the average output power 
was a linear sum of pulses which could in general overlap. Thus, on the 
average, the guide looked linear in power for digital communication 
applications. We showed that the deviations in a particular guide from 
this ensemble average linearity behavior would be negligible provided 
the input pulses were very narrow compared to the width of the guide 
average power impulse response. How narrow the input pulses had to 
be depended upon how much equalization was required to separate the 
output pulses.t 

It is clear that, since the detector adds the powers in each output 
mode, the total power will be a linear sum of pulses if the individual 
mode powers are. In addition, we may suspect that the deviations from 
the ensemble average in each mode may add randomly while the 
average powers add systematically. Thus, some improvement in the 
signal to “‘noise’’ ratio may accrue from this spatial diversity. 

An interpretation of what is happening to make the deviations 
negligible is the following: Since the guide average power impulse 
response for output mode J, s;(t), is the Fourier transform of the two- 
frequency correlation function S;(w) defined in (26), we can interpret 
the reciprocal of the width of s,(t) as the bandwidth difference over 
which the guide transfer function between the input field and the 
output mode j becomes uncorrelated. When we use narrow input 
pulses compared to the width of the average power impulse response, 
we use a lot of bandwidth compared to this correlation bandwidth and 


t Remember that the output pulse shape becomes independent of the input pulse 
shape as the input pulses get narrow. 
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thus obtain frequency diversity. As the input pulses become very 
narrow, the output pulses become fixed in shape equal to s,(¢), but the 
diversity keeps increasing, resulting in averaging out of the deviations 
from one guide to the next. 


Vv. CASE IV: INCOHERENT SOURCE, MULTIMODE FIBER WITH MODE 
COUPLING 


In this example, we consider an incoherent intensity-modulated 
source exciting a multimode fiber with random mode coupling. We 
assume that material dispersion is negligible. (Since we shall be con- 
sidering a wideband source, we may question the physical reality of 
neglecting material dispersion. On the other hand, the qualitative 
results to follow may provide insight into more general cases.) To 
simplify what will prove to be a somewhat complicated analysis, we 
shall consider the response in a particular guide mode, j, at the output, 
to the field in a particular mode, k, at the guide input. Extension to 
consideration of the total input field and the total response should be 
straightforward, using the techniques outlined below. 

The input field complex amplitude in mode k is of the form 


exist) = Vm(t)e(t), (41) 
where m(t) is the modulation and c(¢) is the optical incoherent carrier 
which satisfies 

(c(t)c*(u)) = R(t — u) (42a) 
(c()c*(u) c(t’) c*(u’) ) 
=R(¢(-—wk.(t’ —~w) + R(t — w)R(t’ — u). (42b) 
The complex amplitude in mode j at the output due to the input 
field in mode k is given by 


cial) = f enall!haelt — Dae, (43) 


where the impulse response coupling mode k to 7 is assumed to satisfy 
the same statistics as were outlined in the first paragraphs of Section 
IV, Le., 
(@) Hw) = F{hjO} 
(it) (Hj.(w) Hew + ¢)) = Si(o) 
(tit) (Hj(w) Haw + o)Aj(o + 0) Hw + 0”) (44) 
= Syx(a)Sj(o” — 0’) + Six(o”)Sjx(o — 0°) 

sot — v’), 


I 


(iv) (hin Apt’) 


LINEARITY AND EQUALIZATION 1191 


where 
s(t) = FS; (w)}. 


The average input power (averaging over the source fluctuations) is 
(Dit) = m()R(0) = [XL arhin(t — kT) JR.(0), (45) 


i.e., a sequence of on-off modulated pulses where a; = Oor1, 7 = pulse 
spacing. 
The output power is given by 


Piou(l) = f [Nm EVV EJolee*l'hiell — Unt — eae'ae”. (46) 


If we average over the source fluctuations and the guide statistics 
we obtain, using (44) and (6) 


(Pinal) = f mUYReW — U)Ahjalt — UDialt — 0") pata 


= [m@)RO)sn(t = eat! = Larhnalt— kT), (47) 


where 


iu (t) = Re(0) / hin(t’)sju(t ~ t’)dl’. 


Thus, on the average, the output power looks like a linearly filtered 
version of the input power with impulse response s;z(t). 

This average output power will produce an average detected- 
equalized voltage which is also a linearly filtered version of the input 
power. We next show that, if the optical bandwidth of the incoherent 
source is sufficiently large, then the deviations of the detected equalized 
voltage from its average, because of fluctuations in the source and 
deviations of the impulse response of a particular guide from the 
ensemble average, can be neglected. 

As before, the detected-equalized voltage is given by 


Vout (t) = z f Dowlthacs.-tie.¢ rs t') dt’, (48) 


where 2 is an arbitrary constant. 
The mean detected-equalized current and the mean square deviation 
from the mean are given by 


(our (t)) = 25 Anh oou(t = kT), 
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where 
oot) a: [rina Yhact.-1n.(¢ ro t') dt’ 


o3(t) = (veue(t)) — (our(t))® = f Pinull)Pioull”)) 
X haet.-tis.(6 — U’)haet.-tir. (6 — U’)dé'dt” — Wour(t))? (49) 


To calculate the mean square deviation we need the correlation func- 
tion of p,,,,(). Using (6), (42b), and (44) we obtain the following com- 
plicated expression: 


(Dourlt’) Djoult’’)) = (Djoarlt’) MD iourlt’’)) | : ole eran 


+f Lolz) THsze(e — 7) Sjn(t"" — n)dr f | Re(u)|*du 


+ f Crm(r)PEm(e” — ef = 2) Blane! — 2) 8dr 
x / \R.(u)|?du. (50) 


In order to obtain numerical results we must make some assumptions 
as to the shapes of s;(t), h,(), and R.(t). We shall assume the 
following: 


h,,(t) = exp —#?/2c2 (Gaussian-shaped input pulses) 
F{R.(t)} = exp —w?/2B2 (Gaussian-shaped source spectrum) 


Sy(t) = exp —t?/2y? (Gaussian-shaped power impulse re- 
sponse—appropriate for long guides). 


Il 


(51) 


Using (51) in (50) we obtain the expressions 


B 
(pj @)) = >> azel—t- #2 )2/2(08+72)] oyDs 
ou x aopriay 
(Dour (z’) Piout (t’’) ) 


= Pint!) )(Pinw(l”)){1 + exp [—(” — 1) B32] 
Boy . 
+ FDS age 02 42 e- LA) T2402 
QV? + 7? : 
x gl-lotty2] (t/+[ (t//—t") /2]—(k+ 5) (7 /2)}2 +. enka) T+ (08! ) 12/402 
Kelrlerty2(e+ let") /2]-(e+9) (1/2))2) (52) 


Equation (52) is still fairly complicated, but we can make some 
general observations. The mean output power is proportional to the 
source bandwidth B,, and therefore the mean output power squared 
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will be proportional to B?. Looking at the mean squared output power, 
we see that the portions involving the double sum are proportional to 
B,. Those terms will become negligible compared to the mean output 
power squared as B, gets large. The term exp —(t’’ — ¢’)?B? can be 
approximated by an impulse of area V2r/B,. Thus, this term is also 
proportional to Bs! and will be negligible for large B,. Thus, for large 
B, we can approximate the mean square power by the mean power 
squared. Equivalently, o2(¢) will become negligible compared to (Uout(¢)) 
as the source bandwidth becomes very large. Just how large the source 
bandwidth has to be depends upon how much overlap there is to 
average output pulses and therefore upon how much equalization has 
to be done. In the simple case where the output pulses do not overlap 
and where the equalizer is matched in shape to the output pulses, we 
obtain 


S_ Worl) NB, gy 
BOOS oe Me St, NIE, Ne 


for 


T > 3vo? + y? = 1.5 X output pulse width 
haet.-tit.() = hy,.,(—f) (matched filter). 


From (53) we see that, in this special case, the ratio of the mean 
detected equalized (filtered) voltage squared to the mean squared 
deviation, because of source fluctuations and guide random coupling, 
will be greater than 1000 if the optical source bandwidth is more than 
1000 times the reciprocals of both the input pulse duration and the 
average power impulse response duration. 

Typically, the source bandwidth is more than 10! radians per 
seconds. Thus, the deviations are negligible for input and output pulse 
widths larger than 10-!° seconds. Of course, more bandwidth is re- 
quired to make the deviations negligible when there is considerable 
pulse overlap. 

An interpretation is similar to previous cases. The requirement that 
the source bandwidth be large compared to the reciprocal of the 
duration of the input pulses allows averaging out of the fluctuations in 
the source. The requirement that the source bandwidth be large com- 
pared to the reciprocal of the guide average power impulse response 
duration gives the frequency diversity that averages out the deviations 
between guides resulting from random mode coupling. 
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VI. CONCLUSIONS 


We conclude that there are a number of interesting circumstances in 
which a fiber system, normally linear in voltage, can be considered 
linear in power for digital communication purposes. The input power 
consisting of a sum of on-off modulated pulses produces an output 
power (and thus a detected current) which is also a sequence of on-off 
modulated pulses, possibly overlapping. The output pulse shape is 
well-defined under the conditions described above, which usually 
amount to using enough optical bandwidth to have sufficient fre- 
quency diversity to average out source fluctuations and/or random 
mode coupling differences between guides. This power linearity allows 
baseband equalization (with the usual noise penalties) to allow use of 
the guide at higher bit rates than would be associated with the criterion 
that the output pulses must not overlap. Many assumptions made 
above may not be completely applicable to particular guides, but it is 
hoped that, qualitatively, some insight as to when power linearity may 
occur will be derived from these results. 
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Optimum Network Call-Carrying Capacity 


By R. L. FRANKS and R. W. RISHEL 
(Manuscript received September 8, 1972) 


A telephone network with switching and trunk congestion is considered. 
An optimization problem expressed in terms of mean numbers of calls and 
mean rates of flow of calls in various categories of service throughout the 
network is formulated. The maximum mean number of talking calls given 
by this optimization problem is an upper bound on the mean number of 
talking calls which could be carried by the network using theoretically 
optimum network management. Examples are given suggesting that the 
upper bound is close to values which actually can be attained. 

The optimum of the problem is achieved by controls which (2) restrict 
the number of calls coming into the network from the end offices and (it) 
route appropriate fractions of the remaining calls over the various possible 
routes. 


I, INTRODUCTION 


Telephone communication facilities are designed to adequately 
handle peak traffic loads of an average day. In many instances the 
system is subject to higher loads. Classic examples of situations in 
which overloads occur are during holidays such as Christmas or 
Mother’s Day, after disasters such as earthquakes or hurricanes, and 
during facility failures. Because of the time lag necessary to install new 
equipment, high overloads can also occur in normal operation in cases 
in which predicted traffic growth is greatly exceeded by actual traffic 
growth. An interesting observed phenomenon is that under certain 
high-load situations fewer calls may be completed than during normal 
load periods. Recognition of this gave rise to the subject of network 
management. One objective of network management is to control the 
handling of calls so that the maximum number of calls is put through 
the network. 

An interesting discussion of the network management problem and 
early work to understand the phenomena involved in it is contained in 
Ref. 1. In response to the problems mentioned in Ref. 1, a simulation 
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study of the network management problem was carried out in Ref. 2. 
Simulations? of a modest-sized telephone network were developed and 
were used to evaluate the performance of various control techniques. 

A large number of control techniques have been suggested for 
managing the network under overload conditions. Some papers which 
are representative of these controls are Refs. 3 through 5. 

For given point-to-point calling rates, and given network manage- 
ment controls, let us measure the steady-state performance of the net- 
work by the expected total number of talking calls carried by the 
network. Define the capacity of the network (with the given calling 
rates) to be the maximum performance that could be obtained by any 
network management controls. Our objective is to set up techniques 
for computing the capacity of the network. The capacity of a network 
can be used as a benchmark in evaluating network management 
controls. That is, the performance of a given network management 
control system could be computed and compared against the capacity 
to tell how effective the given control system is. 

The calls carried on a telephone network can be considered as a 
stochastic process described by a very large number of variables. There 
are techniques which apply to optimization of stochastic systems;*°® 
however, these would not give practical methods for computation of 
the optimum capacity. 

For simplicity, we will consider only the steady-state situation. Our 
method will be to establish inequalities and equations which must be 
satisfied by the mean steady-state values of numbers of calls in various 
categories of being set up and mean rates of flow to calls into and out 
of these categories. From these equations and inequalities an optimiza- 
tion problem will be formulated. It will not be claimed that the set of 
inequalities and equations obtained is an exhaustive set. Hence the 
optimum value of the criterion of the optimization problem will only 
be an upper bound on the optimum value for a corresponding criterion 
for the real system. Later, examples will be given to show that in these 
cases the optimum value of the optimization problem can be nearly 
obtained by an appropriate choice of controls incorporated in a simula- 
tion of the network. 

While the situations are quite different, our treatment of this problem 
follows in spirit a corresponding technique in optimal control theory, 
called “relaxation” of the problem.!*:1! However, the interest there is 
in finding an optimal control, while we are not after an optimal control, 
but merely want a good upper bound for the message carrying capacity 
of the network. 
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II. BACKGROUND 


The factors underlying the decrease in the number of calls carried by 
the telephone network as it became highly overloaded were already 
well understood in the early work of Ref. 1. As a call is being set up, it 
uses equipment in one switching machine until the next switching 
machine on its route accepts the call and receives the destination of the 
call from the previous machine. If a switching machine becomes over- 
loaded, machines adjacent to it will have to wait longer to have their 
calls accepted and the destinations passed on. This causes an increase 
in the service time for putting a call through these machines. This in 
turn may cause the adjacent machines to become congested. A current 
device to relieve this congestive phenomenon is that calls that must 
wait longer than a fixed time-out time for a subsequent machine are 
given a no-circuit announcement. However, even with time-outs, 
switching machine congestion can back up throughout the network. 

Calls being set up occupy trunks on the partial route over which 
they have progressed. If a large number of calls are attempting on 
routes which are blocked, a portion of the capacity of certain links 
could be used by these ineffective attempts trying to set up. This would 
use capacity that could be used by talking calls. These ineffective at- 
tempts also use switching capacity in machines preceding the blockage. 
Most blocked attempts try again. These retrials increase the congestion. 

The model which will be set up will incorporate the features men- 
tioned above. Based on these observations, the model must take into 
account both trunking congestion and switching machine congestion. 
Since the route a call may take can be controlled and calls which are 
given busy signals free the entire partial route they occupied, the model 
should keep track of the partial routes over which calls have progressed. 

Throughout the entire paper we will be interested only in the ex- 
pected values of the various variables in the steady state. We will 
assume throughout that the processes are ergodic. 


III. SWITCHING MACHINE MODEL 


A block diagram of the operations of the switching machine which 
will be modeled is given in Fig. 1. This is a simplified model of the 
Bell System No. 4A-ETS switching machine. In the model a call 
coming into a switching machine enters a queue to wait for a vacant 
sender. When a call gets a sender it inpulses its destination information 
to the sender and releases the sender it had in the previous machine. 

The call then gets into a queue for a decoder-marker combination. 
This decoder-marker decides on the machine the call should be routed 
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Fig. 1—Switching machine block diagram. 


to next, tests for a vacant trunk, and sets up the connection, if possible. 
If there are no vacant trunks to appropriate subsequent machines, a 
no-circuit announcement is given. After a no-circuit announcement, 
the trunks on all the links that the call had progressed through become 
vacant again. If it is routed to a subsequent machine, it enters a queue 
for a sender in that machine. 

The sender in the current machine is occupied by the call from the 
time it begins processing the call until the call has-transmitted its 
destination information to the sender which processes the call in the 
subsequent machine. If a call waits longer than a fixed time to get a 
sender in the subsequent office, it is timed out. If it is timed out, the 
call is sent back to the marker-decoder which then connects it to a 
no-circuit announcement. 

The process of a call being connected for service to a sender is 
accomplished by a sender link controller and a sender link connector. 
These devices test for an idle sender and an idle path to the sender. 
Then a path through the switch from the incoming trunk to the sender 
is selected and the connection established by closing appropriate 
switches. 

The process of connecting a call for service to a marker is carried out 
in a similar fashion by a marker connector. 
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IV. COMPLETE AND PARTIAL ROUTES 


A telephone network can be thought of as a collection of call switch- 
ing machines connected by communication links. In the operation of 
the network, a talking call which hangs up frees a trunk on all the links 
of the route it occupied. A call which is in some state of being set up 
will occupy a partial route. If it is given a no-circuit announcement, it 
will free trunks on all the links of the partial route it occupied. 

To model routes and partial routes, we will make the following 
definitions. Assume there are k switching machines of the network 
labeled by the integers 1, --- , &. We will use the letter & to denote a 
complete route. A complete route 


R= (ta, +++, tn) 


is a succession of switching machines. A call occupying a complete 
route is always considered to be a talking call and it is understood that 
the call is occupying trunks on the links connecting switching machines 
in adjacent positions in the expression R. 

A partial route describes the route occupied by a call in the process 
of being set up and the destination of the call. Let r designate a partial 
route. If 

r= (41° * “tn-l, dale 


it is understood that the call has passed through machines i1-- -7n—~2, 
is waiting to get into 7,n_1 or is in 7,_1 being processed, and its destina- 
tion is machine 7,. It occupies trunks on links connecting adjacent 
machines from 12; to 7,-1. When a partial route r is of the above form, 
we shall say the partial route ‘‘terminates”’ at machine 7,,_1. 

The symbol r* will be used to designate a route subsequent to r. 
If r = (t1--+tn_1, tn), rt may be the complete route 


R= (t1° + +tn—1, tn) 
in the case in which there is a link between machine 7,_1 and the 
destination 7,; or it may be a route of the type 
gt = (41,°¢°° , tn—1, Em; tn), 
that is, a partial route in which the call has passed to one further 


machine 7, on its way to its destination 7, than the call on partial 
route r had. 


V. MODEL DESCRIPTION 


In a given switching machine, calls can be distinguished by the 
partial route they occupy and the stage of processing they are in. 
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Completed talking calls can be distinguished by the complete route 
which they occupy. To describe the operation of the switching 
machines, we define the variables: 


W, = mean number of calls on partial route r waiting for a sender 
S, = mean number of calls on partial route r which are inpulsing 
into a sender 
V, = mean number of calls on partial route r waiting for service by 
| a decoder-marker 
_M, = mean number of calls on partial route r being serviced by a 
decoder-marker 
Nr = mean number of talking calls on complete route R 
x, = mean rate of flow of calls on partial route r into the sender 
queue 
z, = mean rate of flow of calls on partial route r into the sender 
v, = mean rate of flow of calls on partial route r from the sender 
into the decoder-marker queue 
u, = mean rate of flow of calls on partial route r into decoder- 
marker 
t, = mean rate at which calls in sender queue on partial route r are 
timed out 
b, = mean rate at which calls on partial route 7 are blocked 
2r = mean rate at which calls are being completed through the 
network on complete route R. 


We shall show that the following statements must be satisfied by 
these mean rates and mean numbers. 


1. The mean rate at which calls flow into senders in a given machine 
is less than or equal to a constant times the mean number of 
senders which are not currently processing calls. 

2. The mean rate at which calls arrive at the marker queue from the 
senders is equal to a constant times the mean number of senders 
which are processing calls which have not yet entered a marker 
queue. 

3. The mean rate at which calls flow into the marker-decoder is less 
than or equal to a constant times the mean number of markers 
not currently processing calls. 

4. The mean rate at which calls leave the markers is equal to a 
constant times the mean number of markers which are processing 
calls. 
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Consider statement 1. Let us define a call to be in the process of 
connecting to a sender from the time an idle sender is found until the 
connection has been completed to that sender. Defining the process 
this way, no time-outs occur during it. While a call is undergoing this 
connection process, we will say it is in the connector. We use this 
definition: 


number of calls connecting to some sender 
< number of free senders. 


The above inequality also must hold for the mean values of both 
quantities. 

Now the number connecting to some sender is the number in a 
queuing system (the connector) whose service time is the time required 
to make a connection. Applying Little’s Theorem!*:13 gives: 


E {number of calls connecting to some sender} 
= (mean rate at which calls are flowing into the connector) 
X (mean time to make a connection). 


Since we are considering a steady-state situation, the mean rate at 
which calls are flowing into the connector equals the mean rate at 
which calls are connected to some sender. Hence we obtain statement 
1. The constant in statement 1 is the reciprocal of the mean work time 
required to connect a call to a sender. 

Consider statement 2. Calls attach to a sender, impulse their destina- 
tion information, and then enter a marker queue. Applying Little’s 
law to the number of calls inpulsing into a sender gives, 


mean number of calls inpulsing into a sender 
= (mean rate at which calls arrive at senders) 
X (mean inpulsing time). 


Since the system is in steady state, the mean rate at which calls arrive 
at senders equals the mean rate at which calls arrive at the marker 
queue. The mean number of calls inpulsing into a sender is the mean 
number of calls in the sender which have not yet entered the marker 
queue. Hence statement 2 is established. 

Statements 3 and 4 are statements concerning markers similar to 
statements 1 and 2 concerning senders. They can be established by 
using Little’s law in a similar manner to statements 1 and 2. 

Next, statements 1 through 4 will be expressed more formally as 
equations and inequalities in the variables defined previously. For a 
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given switching machine, let 


I; 
0; 


I] 


all partial routes terminating in machine 7 
all partial routes immediately subsequent to a partial route 
terminating in machine 7 
S; = total number of senders in machine 7 
9; = total number of decoder-markers in machine 7. 


The total rate at which calls are entering senders in machine 7 is the 
sum over all the partial routes which terminate at machine 7 of the 
rates at which calls on those partial routes are entering senders. In 
symbols this is 


emis 


rely 


The mean number of calls occupying senders in machine 7 is 


> S+V-+M) + XO Wt. 
r&l; r+ 0: 


Hence, the mean number of free senders is 
8i — | 3 (S, + V,+ M,) = = Wel. 
rGli r+ Eo; 
Hence, statement 1 may be written as the inequality 
>» 2 S Cx|8:— > (S, + V,+ M,) - ~~ w.'} (1) 
rely J reli r+G0z 


A similar interpretation of statement 3 yields 


yous Cs} -~= m,|. (3) 

Statements 2 and 4 are expressed by 
v, = CS,, (2) 
ue = CuM,. (4) 


The number of calls on a given link either talking or being processed 
in some switching machine must be less than or equal to the number of 
trunks in that link. Hence, the expected number of calls of these types 
must be less than or equal to the number of trunks on the link. The 
inequalities expressing this are given by 

Dw Net > (W,+8,+ V.+ M,] S Ci. (5) 


RIi,j rDi,7 
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In this notation there is one inequality for each link connecting a 
machine 7 and a machine 7, C’;; is the number of trunks on this link, and 
the notations R D i,j and r > 1,j indicate that the sums are respec- 
tively over all complete routes or all partial routes which pass through 
link 7,7. 

In the steady state the expected rate at which calls flow into any 
category of processing in a switching machine must equal the rate at 
which they flow out. Thus the following flow-in equal flow-out equa- 
tions must hold. 


t,=2,+t, (6) 
2, = Uy (7) 
++ > tif =4, (8) 
rt Co; 
u,= D> a,+4+ 5, (9) 
r? G0; 


Let Ag denote the probability that a call that completes through the 
network on route RF will be answered by the customer. The rate at 
which calls are completing to talking calls on route R is Azza. If the 
mean length of a talking call is 1/v, calls will be hanging up at rate v. 
To be in steady state, the equation 


Arzr = pN R (10) 
must hold. 

Let \;:; denote the mean rate at which calls wish to enter the network 
originating at machine 7 with destination machine j. Suppose that, 
if a call is placed and receives a no-circuit announcement, the customer 
decides to retry with probability P or to give up placing the call with 
probability 1 — P. Suppose this is true independently for every call 
irrespective of how many times the customer may have tried pre- 
viously to place the call and failed.* 

Let a;; denote the rate at which calls including retrials are being 
placed from 7 to 7. Let 7;; denote the rate at which no-circuit announce- 
ments are being given and s,; the rate at which calls are being com- 
pleted from 7 to 7. Then 


Qiy = Si; + rij = Nay + Pr;;. (11) 


*Customer retrial behavior is discussed by Wilkinson in Ref. 14. The model 
considered here can be considered as an idealized approximation to the more com- 
plicated behavior reported in Ref. 14. 
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Solving for 7;;, 
Ng — Say 
aan 
Now the rate at which calls are being completed between 7 and 7 is the 
sum over all the complete routes joining 7 and 7 of the rate at which 
calls are flowing onto these complete routes. Hence, 
sii = a ; ARér. (18) 


R=(i, eee,j) 


(12) 


Using (11), (12), and (13) gives 


Ay—-P > Aner 
a R=(,...,3) : 
Qj i_ Pp (14) 
Calls entering the network at machine 7 will be assumed to originate 
through an end office which leads into machine 7. Since this is so, there 





~ 2 S$ C{Si- > (S-+V-+ M,) — 2%, W,t} (2.1) 


rel: rel: 


~~ Ur < C3{M, = Ss M,} (2.2) 
rel: reli 


REaG9+43 
(4,3) = ———#eG) bg (2.3) 


b> Nret+ 3 (We Pas Va) = Cy (2.4) 


RDi,j rDi,j 
v, = CS, (2.5) 
u, = CM, (2.6) 
Ly = 2, + ty (2.7) 
Zp = Uy (2.8) 
Artr = wNe (2.9) 
Ur = 0p + mo: i,t = Hy, xyt + 0b, (2.10) 


Fig. 2—Equations and inequalities described in Section V. 
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will be a possibility that a call which wishes to enter the network at 
machine 7 with destination machine j may be blocked in the end office 
prior to its getting into the network. Let b(;,;, denote this rate at which 
calls from 7 to 7 are blocked in the originating end office. 

If r = (2, j), that is, r is the partial route of a call just starting at 7 
whose destination is 7, then the rate of flow onto r is given by 


Lr = Aiz — 5G,3). 


Since all the variables of the problem are mean numbers of calls 
or mean rates at which calls are flowing, all variables must be 
non-negative. 

The equations and inequalities which have been described so far are 
gathered together in Fig. 2. 

Rewriting the equations of Fig. 2, it can be seen that z,, v,, u;, Nr, 
S,, and M, can be expressed in terms of z,, ¢,, and b,. Eliminating these 
variables from the problem, we arrive at the equations expressed in 


Fig. 3. 
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Fig. 3—Formulas of the relaxed optimization problem. 
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VI. AN OPTIMIZATION PROBLEM 


Up to this point we have been considering the very complex sto- 
chastic process of calls progressing through a telephone network. For 
any such process, the constraints in Fig. 3 must be satisfied by the 
appropriate mean values. Next we will consider the relaxed optimiza- 
tion problem mentioned in Section I. 

The total expected number of talking calls at a given time is 


» Nr. (15) 


Suppose it is desired to maximize this quantity. From (10) this is 
equivalent to maximizing 


2 Arzr. (16) 


Let us first assume that it is possible to control the variables 
Caer Vig Way DK be 


We formulate the optimization problem of maximizing (16) subject to 
the formulas of Fig. 3. 

This is the relaxed optimization problem mentioned in Section I. 
Notice that while we have argued that the formulas of Fig. 3 must 
hold, it seems apparent that other relationships than those given in 
Fig. 2 will have to be satisfied for corresponding variables in the real 
network. However, we shall treat this relaxed optimization problem as 
a “model” for the network and investigate the optimum mean number 
of calls carried by this model. Later, for an example, the calls carried 
by the model and those carried by a call-by-call simulation will be 
compared to show that these call-carrying capacities are close. In this 
sense, this indicates that the model mirrors the major features of the 
call-carrying capacity of the network. 

Notice that the variables which are to be controlled contain mean 
rates of flow of calls throughout the network. Implicit in this is the 
assumption that the routing of calls is to be chosen in the relaxed 
optimization problem. This will result in a routing different than 
conventional alternate routing. The relaxed optimization problem as 
formulated is a linear programming problem. If rules for alternate 
routing were imposed in addition to the formulas of Fig. 2, this linearity 
would be destroyed. For this reason the more flexible type of routing 
will be allowed rather than insisting on conventional alternate routing. 

Notice that it might be felt that the variables to be controlled by 
the program are not really subject to control in a real network. It will 
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be shown that the optimum solution only adjusts a subset of those 
variables and it does appear that the variables which must be adjusted 
in the optimum solution are subject to adjustment in a real network. 


VII. GRADE OF SERVICE CONSTRAINT 


In the relaxed optimization problem that has just been formulated, 
no provision has been made to assure maintenance of an appropriate 
level of service. For instance, it is conceivable that the solution of 
the optimization problem would deny service completely to some 
point-to-point pair if the facilities on the route it used could be better 
utilized by other traffics. A provision should be made to insure at 
least a certain level of calling between each point-to-point pair. 

Since the situation to be considered is one in which the network is 
already overloaded, it will not be possible to keep the blocking for 
each point-to-point pair below desired levels. However, it is possible 
to require that each point-to-point pair has the capability of com- 
pleting through the network a fixed minimal number of calls per unit 
time. The total mean rate at which calls are completed through the 
network between point-to-point pair (7, 7) is 

eR 
R=(i,-+ +4) 
The inequalities 
2R = Ki; (17) 
R= (a, eee J) 
assure that calls between each pair (2, 7) will be completed through the 
network at a mean rate greater than or equal to K;;. In the future 
inequalities, (17) will always be added to the formulas of the relaxed 
optimization problem given in Fig. 3. 


VIIT. REDUCTION OF THE OPTIMIZATION PROBLEM 


Notice that in this problem if zz, z,, b,, t,, V+, W, is an optimal 
solution, then there is another optimal solution with the same zz and 
z,, the same b, for r not of the form r = (77), new b(zj) equal to the 
previous b(7j) + ¢(zj), and with V, = W, = ¢t, = 0. This is so since 
the equations and inequalities are still satisfied and the quantity (16) 
to be maximized is unchanged. 

Notice also that eq. (3.1) in Fig. 3 implies that 

er = > 2R- (18) 


rR 


In this, the notation r > R means that the sum is to be taken over all 
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complete routes which have the partial route r as their common 
beginning and destination. 
Let 
Zr, @r, 6, 0, 0, 0 


be any optimal solution of the type described above. If 
b= 0, rel, A), 


bay = E cat <a > Anze|C1 se dl pa meter zr, (19) 


R(t, + +47 I) 
rOR 
Then 
ZR, Zi, b,, 0, 0, 0 


is an optimal solution since (20) implies equation (3.1) in Fig. 3 is 
satisfied, (18) and (19) imply that the equations or inequalities (3.2) 
through (8.5) are satisfied, and (16) and (17) are unchanged. This 
implies that the optimization problem may be rewritten in terms of 
only the variables b(;,;) and zz through using eqs. (19) and (20). The 
optimization problem is restated in this form in Fig. 4. 


IX. CONSEQUENCES OF THE OPTIMIZATION PROBLEM 


Notice that the variables of the final optimization problem are 
bo; = the fraction of calls given a no-circuit announcement in 
their originating office 
Zr = rate at which calls are being completed through the network 
on complete route & 


and that it has been shown that if b;;;, and zz are optimal values for 
the final relaxed optimization problem, then 


@r,2@r = >, @r, Vr =0, W,=0, t, =0 


Ror 
: be,3) if r= (2, 9) 
"lo if r# (i, 9) 


are optimal values for the original optimization problem. 

This implies that the solution of the relaxed optimization problem 
may be taken to be of the following form: An appropriate fraction of 
point-to-point attempts are blocked in their originating end office. The 
remainder is appropriately divided among the various complete routes 
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Maximize 
X ArZr (4.1) 
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> (l-—P+ PAg)er = dj — (1 — P)buy (4.4) 


R=(,- 7 2) 


ae Ky (4.5) 
R=(t,-°++,9) 


Fig. 4—Reduction of the optimization problem. 


joining the origin and the destination. No further blocking takes place. 
The fractions of traffic assigned to each complete route are chosen so 
that no queues build up in senders or markers and no blocking occurs 
on trunks. They also maximize the total expected number of talking 
calls. The fraction of each point-to-point traffic which is blocked at its 
source and the fractions which are routed over each complete route 
joining a point-to-point pair can be considered to be the optimal 
controls chosen by the relaxed optimization problem. 

Of course, the above paragraph refers to the solution of the relaxed 
optimization problem and not to an optimal control for a real network. 
Recall that the relaxed optimization problem contained some, but not 
all, of the constraints of a real network. As a result, its optimal solution 
may violate some of those additional constraints. However, the message- 
carrying capacity found by the relaxed optimization problem must be 
at least as large as the capacity of the corresponding real network. 

The optimal solution of the relaxed problem is suggestive of a good, 
but suboptimal, control for a real network: Code block calls in the end 
offices by the same amount as used in the relaxed optimization problem 
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and route the calls as indicated. Do not intentionally block calls at any 
other point. Since the real system is stochastic, this will result in some 
queues forming and, depending on the method of implementation, in 
some blocking internally in the network. Because of this, the control 
might be improved by choosing slightly different values of code 
blocking. 


X. JUSTIFICATION OF THE OPTIMIZATION PROBLEM 


It is natural and crucial to ask if the capacity of the model will 
approximate the capacity of the stochastic network. It might be 
thought that the optimal solution of the model is an ‘‘Alice in Wonder- 
land” solution due to the relaxed nature of this model, especially since 
it chooses just the right amounts of traffic to route over each complete 
route so that there is never any internal blocking or any queues in the 
switching mathines. We will try to demonstrate that this is not so by 
the following argument. 

It has been shown in establishing the equations and inequalities of 
Fig. 4 that corresponding variables for any real network must satisfy 
these conditions. Hence, these variables for a real network are a feasible 
set for the optimization problem. Thus the capacity of the model is an 
upper bound on the capacity of a real network. We will show in typical 
examples that controls similar to those of the optimization problem 
when incorporated in a stochastic simulation of these networks give a 
carried load close to this upper bound. 

The first example is shown in Fig. 5. The small network shown there 
was subjected to severe overloads. An overload factor of one corre- 
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MESSAGES IN PROGRESS 





OVERLOAD FACTOR 


Fig. 5—Comparison of upper bound and achieved carrier loads with severe overloads. 
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Fig. 6—Network used in focused overload example. 


sponds to the traffic load for which the network was designed, an 
overload factor of two corresponds to twice the design load, and so 
forth. For all the overloads shown the upper bound on the network 
capacity was 238 messages. A Monte Carlo simulation of the network 
was run restricting the offered traffic at the end offices. The steady-state 
average number of messages carried was 228. 

The second example was run on the considerably larger network 
shown in Fig. 6. This network received its design offered load except 
that all offered traffics destined for the node marked ‘‘Focus’’ were 
eight times their design levels and all traffics originating at that node 
were twice their design levels. Focused overloads of this type occur 
in the toll network. A similar pattern might occur following a natural 
disaster in the vicinity of the node marked ‘‘Focus.”’ 

Four cases were run on this network with this offered load. In all 
four cases only in-chain routing was used, i.e., only message paths 
which could exist under the current hierarchial routing scheme were 
allowed. The results are shown in Fig. 7. First, a Monte Carlo simula- 
tion was run using short sender timing as the only network manage- 
ment control. The average number of messages carried in steady state 
was 250. Second, the simulation was run using the network manage- 
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Fig. 7—Comparison of upper bound and achieved carrier loads with focused overload. 


ment controls proposed for the No. 4 ESS switching machine, the next 
generation of Bell System toll switching machines. The steady-state 
average number of messages carried by the network was 1239. Third, an 
analytic model!® was run with code blocking, not in the end offices 
but in their associated toll machines. The steady-state average number 
of messages carried was 1273. Finally, the upper bound on the network 
capacity was 1329. 

In this example the proposed, economically feasible No. 4 ESS 
controls achieve most of the improvement between the essentially 
uncontrolled case and the upper bound on network capacity. The upper 
bound is more nearly attained by choosing code blocking in the toll 
centers, based on knowing the mean offered loads. Presumably the 
upper bound could be even more closely approached if code blocking 
was done in the end offices and some routing controls were included. 

In summary, not only is it likely that controls based on complete 
knowledge of the underlying distributions can nearly achieve the 
upper bound on capacity, but also economically feasible control 
schemes can approach it. 


XI. SOLUTION INFORMATION 


Computer programs have been set up to solve the final linear pro- 
gramming problem. The program consists of two parts. The first part 
takes given data on the network, such as number of toll centers, which 
toll centers are connected by links, number of trunks on a link, and 
machine operating constants. Then it writes the equations given in 
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Fig. 4 in a form suitable for use in a linear programming algorithm. The 
second part uses a linear programming algorithm to solve the linear 
programming problem. These programs have been used to compute 
optimal controls for moderate-sized networks. 

For a network with 17 nodes, 57 links, and 272 point-to-point 
offered traffics, it cost about $45 to run both programs on an IBM 3860. 
For 21 nodes, 204 links, and 441 point-to-point offered traffics, it cost 
about $100 to run both programs on the same machine. By modifying 
the standard linear programming algorithm, taking into account the 
special structure of the model, it appears that it will be feasible to 
compute capacities for very large networks. 

The output of the solution algorithm for the linear programming 
problem contains much additional information which can be important 
in evaluating the network’s operation. The amount of slackness in the 
inequality constraints and the optimal variables for the dual linear 
program are printed out. 

In Fig. 4, the left side of inequality (4.2) is the rate at which calls 
are processed in switching machine 7. The left side of inequality (4.3) 
is the average number of calls on the trunk group connecting machines 
zand j. The left side of inequality (4.5) is the completion rate for calls 
originating at machine 7 and destined for machine /. 

The dual variables for inequalities (4.2) and (4.3) are related to the 
incremental increase in the optimal carried load which could be 
achieved by adding one sender or marker or one trunk in the indicated 
place. The dual variables for inequality (4.5) give the incremental in- 
crease in the carried load which could be achieved by relaxing the 
minimum service constraint for a particular source-destination pair, 7-/. 

The amount of slackness in inequalities (4.2) and (4.3) gives infor- 
mation on how efficiently the toll centers and trunks are being utilized 
in carrying the given traffic. 


XII. CONCLUSIONS 


A steady-state, mean-flow-rate model of an overload telephone net- 
work with trunk and switching congestion was set up and optimized. 
The expected number of calls carried by this model is an upper bound 
for the number of calls which can be carried in the real network. The 
model takes into account the progression of calls along their routes as 
they are being set up and the amount of trunk space and switching 
machine capacity used by the calls which are in the process of being 
set up. The form of the model was such that the optimization could be 
carried out using conventional linear programming. 
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The calls carried by the optimized model can be used as a standard 
against which the calls carried by various network management control 
systems can be compared. The form of the controls of the optimized 
model should provide insight for network management. Computations 
indicate that there can be a significant difference between the number 
of calls carried by an unmanaged system and the optimally managed 
system. Some information which may be valuable in assessing the need 
for additional facilities in the network is available from peripheral 
information supplied by the linear program. 
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Analysis of First-Come First-Served 
Queuing Systems With Peaked Inputs 


By H. HEFFES 
(Manuscript received March 8, 1973) 


This paper treats the problem of analyzing a first-come first-served 
queuing system, in equilibrium, when subjected to a peaked input (e.g., 
traffic overflowing a trunk group with Poisson input). The basic GI/M/N 
(renewal input to N exponential servers) queuing result is used, together 
with each of two models for representing peaked traffic, the Equivalent 
Random (E-R) model and the Interrupted Poisson Process (IPP) model. 
The equilibrium virtual delay distribution is derived and compared with 
the equilibrium distribution of delays seen by arriving calls. Numerical 
examples are presented, along with comparisons of results using both the 
above models. The results show that delays can be quite sensitive to 
peakedness. 


I, INTRODUCTION 


It is well known, from analysis of blocking in trunk groups, that the 
blocking seen by peaked traffic (e.g., traffic overflowing a first-choice 
trunk group with Poisson input) can be significantly larger than block- 
ing seen by Poisson traffic with the same intensity. In this paper, we 
are interested in determining the effect peaked traffic has on delays in 
queuing systems. The analysis was motivated by a study of sender 
attachment delay in Crossbar Tandem switching machines receiving 
alternate routed (peaked) traffic. 

We treat the problem of analyzing a first-come first-served queuing 
system with peaked input for the situation where there is no idle 
server if there is a waiting customer. The basic tool is the GI/M/N 
queuing result which requires a characterization of the input process 
in terms of the Laplace-Stieltjes transform of the interarrival time 
distribution. This characterization is provided using Wilkinson’s 
Equivalent Random! (E-R) model where the peaked input is modeled 
as an overflow process from a finite trunk group with Poisson input. 

1215 


1216 THE BELL SYSTEM TECHNICAL JOURNAL, SEPTEMBER 1973 


The size of the finite trunk group and intensity of the Poisson input are 
chosen so that the overflow process produces the desired mean and 
peakedness (variance to mean ratio of trunks up on an infinite trunk 
group). The E-R method has been widely used in analyzing blocking 
in trunking networks and is considered our basic model when the source 
of the traffic peakedness is from alternate routing. 

A second model, which gives a much simpler characterization of the 
peaked input (both computationally and analytically), is the Inter- 
rupted Poisson Process.? Here the peaked traffic is considered to be the 
output process of a switch, with Poisson input, where the switch is 
opened and closed for independent, exponentially distributed time 
intervals. The parameters of the switch are chosen to match either 
the first two or three moments of trunks up on an infinite trunk group 
with the corresponding moments obtained for the E-R model. 

Comparisons between results using each of the above models are 
presented along with a set of numerical results which show that delays 
can be quite sensitive to peakedness. 

For Poisson traffic, the virtual delay distribution (time congestion) * 
is identical with the delay distribution seen by arriving calls (call 
congestion). This is not the case for peaked traffic. Since in some 
applications measurements form estimates of time congestion (e.g., 
SADR measurements), it is of interest to relate the time and call 
congestion quantities. Numerical results show the sensitivity of the 
relationship to peakedness. 


II. GI/M/N QUEUING RESULTS 


Consider a recurrent input* to an N server queuing system. The 
service times are independent, exponentially distributed with the mean 
service time given by 1/2. The customers are served in their order of 
arrival, and there is no idle server if there is a waiting customer. 

Let F(x) denote the distribution function of the interarrival times. 
The mean interarrival time, 1/2, is given by 


Il -) 

= [ saF@) (1) 

Ae 0 

_ “Time and call congestion are commonly used in trunking analysis (BCC system ; 

i.e., blocked calls cleared). In the delay case (BCD system), we use call congestion for 

the delays seen by arriving calls and time congestion for the virtual delay. See appendix 

for precise definitions. : 
The reader is referred to chapter 2 of Ref. 3 for a more detailed mathematical 

description of these results. 


t The peaked input will be modeled by recurrent processes. For these processes, 
F(0t)=0. 
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and the Laplace-Stieltjes transform of F(x) is given by 


&(s) = i. e-**dF (x). (2) 
We define the load/server (sometimes called occupancy) as 
fares 
~~ Nua’ (3) 


The following result is available to us:3 If p < 1, then the equilibrium 
delay distribution (as seen by arriving calls, i.e., call congestion) exists 
and is given by 





Pr{delay > T] = exp [—Nyue(1 — w)T']. (4) 


l—w 
The exponential delay distribution is seen to be a function of two 
parameters, w and A. The parameter w is the solution, in (0, 1), of the 
equation 

w = [Ny(1 — w)]. (5)* 


For p < 1, this equation is known to have a unique solution in (0, 1); 
furthermore, the solution can be found by successively iterating on 
(5); i.e., 





wir = [Ny2(1l — ai) ] (6) 
with wo € [0, 1). If we define 
&; = D( juz) (7a) 
and 
ee | Gee 7b 
j _ OU 1 pa ®,’ ( ) 


then the parameter A is given by 
Sk es Se Re a 
| 1 ox (“)va-y-a | 
[-37 AGa@-HWG-9) 7 
Thus, given the characterization of the input in terms of ®(s), eqs. 


(6), (7), and (8) provide the means to compute the equilibrium delay 
distribution as seen by arriving calls (4). The mean of the equilibrium 


A= (8) 





*In Poisson traffic, the solution of (5) is w = p. In some cases, to be treated later, 
(5) can be solved in closed form. 
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delay distribution is given by 

A 
Nal = 0)" ” 
In the next two sections we discuss the Equivalent Random and 


Interrupted Poisson models for generating peaked traffic and charac- 
terizing ®(s). 


E{delay] = 


III. EQUIVALENT RANDOM MODEL 


The E-R model treats peaked traffic as an overflow process from a 
finite trunk group with Poisson input. The holding time for the trunk 
group is assumed to be exponential with mean 1/y:.* The number of 
trunks and the intensity of the Poisson traffic are chosen so the mean 
and variance to mean ratio (peakedness) of trunks up on an infinite 
overflow group closely match the desired mean and peakedness. If we 
denote the desired mean and peakedness by mg and 2g respectively, 
then Rapp’s formulas‘ for the Poisson load (dq) and number of 
trunks (c), 


eq = ns = Mded + 32a(Za = 1) (10) 
1 
7 _Matze \_ 4 _ 


yield an overflow process which approximates the desired process. It 
should be noted that use of (10) and (11), or truncation of c from (11), 
will often lead to overflow traffic with mean and peakedness different 
from (but usually close to) the desired values. To quantify this effect, 
the actual mean and peakedness should be computed using 


M = AeqB(C, Gea) (12) 


ob es... a 
e= [1 m+ |, (13) 


where B is the Erlang B function. 
Takacs’, Chapter 4, shows that the Laplace-Stieltjes transform of the 
interarrival time distribution of the overflow traffic is given by 


a0) = © (5) 5 Tt + im) /S (47) e+ tw, (4y 


Nea f= 


“Note that we can have y ~ uz. For example, if overflow traffic from trunks is 
offered to a group of senders, 1/u: is of the order of minutes, whereas 1/y2 is of the 
order of seconds. 

The value c is considered an integer. In practice, we round c given by (11) up 
and down and choose the one that gives an actual z closest to zg. The actual m and z 
are computed from (12) and (13) using the rounded values of c. 
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where the empty product is unity. Note that ®(s), given by (14), has 
to be repeatedly evaluated in (6) for the solution of (5) and in (8). In its 
present form, (14) is unsuitable for computation (for large c) because 
of numerical problems. In order to avoid these problems, we use a 
recursive method for the evaluation of 6(s) developed by A. Descloux.® 
If we use the notation *(s) to denote the dependence of £(s) on c, 
then ¢(s) satisfies 





[et =- +1+ 24-2 parry) (15) 
eq eq eq 
with initial condition 
de 
1 aie Ae ee (16) 


The solution is thus obtained as follows: Using (15) to evaluate 
&(s), we iterate on (6) to find the w parameter and subsequently the A 
parameter (8). Having evaluated the w and A parameters we can 
compute Pr[delay > T'] from (4). 

We now turn our attention to the interrupted Poisson process model 
for generating peaked traffic, the resulting simplifications, and com- 
parisons with the E-R model. 


IV. INTERRUPTED POISSON PROCESS MODEL 


This model, suggested by W. S. Hayward and analyzed by A. 
Kuczura,? treats peaked traffic as a Poisson process modulated by a 
random switch where the switch is opened and closed for independent, 
exponentially distributed time intervals. The importance of this process 
is that it can provide a simple and accurate approximation to overflow 
traffic. 

This model contains three parameters, the intensity of the Poisson 
process into the switch, \, calls per second; the mean open time of the 
switch, 1/a, seconds; and the mean closed time of the switch, 1/7; 
seconds. If we choose 


ne =A, = mz+ 8e(z — 1), (17) 
Or Mm = 
ne ae [m + 32 — 1], (18) 
and 
re eae 
7s Vs E 1| Ws, (19) 


then the mean and variance to mean ratio of trunks up on an infinite 
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trunk group, with mean holding time 1/,, will be m and z respectively.” 
This corresponds to the two-parameter match of Ref. 2. The so-called 
three-parameter match is obtained by matching the first three moments 
of trunks up on an infinite trunk group with the corresponding moments 
that would be obtained using the E-R model. In this case, c and aeq are 
computed from (10) and (11) and the switch parameters are obtained 
from 

Ms 








_ - 52(d1 — 50) — do(d2 — 61) 
wa Ser ee (61 — 60) — (62 — 63) ay) 
We _ = 60 As Ca, eq 1 
ao = 7 | j= bi | (21) 
and 
Hh te 23 8 Aa fet | 
1 Ys eq 5o ? (22) 
where the 6;, defined in Ref. 2, are given by 
do = B(C, Geq) (28) * 
and 
i) = oth tes i = Ox—1" (24)* 


For a given mean m and peakedness z, it has been shown? that the 
Laplace-Stieltjes transform of the interarrival time distribution of the 
output process of the switch is given by 


= Karipa Kot opi 
a S++ rip. SH Pops’ (25) 


where the parameters ri, ’2, ki, and kz are given by 


531A + Ws + Vs + V(As + Ws + vs)? cs. 4A 4ws | (26) 


ry = 

fo = 3LAs + Ws ++ Ys V(As + Ws + Vs)” = 4A ws | (27) 
= As — Te 

error oe 

ke =1- ky (29) 


and A, ws, and y, are given by (17), (18), and (19) for the two-param- 
eter match and by (20), (21), and (22) for the three-parameter match. 


* B is the Erlang B function. . 
ae equation can be simply obtained from equation (1.15) in the appendix to 
ef. 1. 
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With (25) defining ®(s), eq. (5) becomes a cubic in the w parameter 
of the delay distribution. Dividing the cubic by the known root at unity 
gives the desired root in (0, 1) 


=. 1+ ai + ae a state) _ 7. ee ) 
oi (FS"* ) \(=3 Nie ae Pe (30) 








with 
_ Tun 
ait Nupe (31) 
and 
_ Tom 
a= Nu (32) 


Thus w, given by (80), together with (8) specify the equilibrium delay 
distribution. Note that the iteration procedure (6) has been eliminated 
and that 4,, defined by (7a), is simple to compute using (25). 


Vv. TIME CONGESTION 


For Poisson traffic, the virtual delay distribution (time congestion) 
is identical with the delay distribution as seen by arriving calls (call 
congestion). This is not the case for peaked traffic. Since in some 
applications measurements form estimates of time congestion (e.g., 
SADR measurements), it is of interest to relate the time and call 
congestion quantities. 

We define time congestion as the delay in being serviced experienced 
by a fictitious call arriving at an arbitrary time ¢ when the system is in 
equilibrium.* It is shown in the appendix that, in equilibrium, the rela- 
tionship between call congestion (CC) and time congestion (TC) is 
given by 


Pr[TC > T] = £ Pr[CC > T] (33) 


if p < 1 and the interarrival time distribution, F(z), is not a lattice 
distribution.? Pr[CC > T’] is the delay distribution seen by arriving 
calls and is given by (4). 


VI. NUMERICAL RESULTS AND DISCUSSION 


One measure of system performance of possible interest is the mean 
delay experienced by arriving calls, given by (9), versus CCS offered 


*See appendix for a more precise definition of time congestion. 

t For Poisson traffic, w = p giving Pr[TC > T] = Pr[CC > T]. 

+The E-R model and Interrupted Poisson model clearly satisfy the nonlattice 
hypothesis. 
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to the servers where 


#CCS = 36 7 m (34) 
2 


with m given by (12) in the E-R model and 





m= Ag, (35) 


We 
We + Ys 
for the interrupted Poisson model. Here Az, ys, and w, are given by 
(17), (18), and (19) for the two-parameter match and by (20), (21), 
and (22) for the three-parameter match. Figures 1(a) and 1(b) are plots 
of the mean delay characteristics for each of the three models of 
interest. The values of peakedness z ranging from 1 to 3 are presented 
on Fig. 1(a), and the z = 4 results are plotted on Fig. 1(b). The param- 
eters of this example are N = 18, 1/y; = 180 seconds, and 1/p2 = 7.6 
seconds. 

It is seen that, while the two-parameter results tend to overestimate 
both the E-R and three-parameter results, the differences are indis- 
tinguishable (for the entire CCS range shown) up to z = 1.5 and small 


—— E-R MODEL 
—— 2-PARAMETER 
—ome me 3—PARAMETER 


MEAN DELAY !N SECONDS 





200 300 400 500 600 700 100 200 300 400 500 600 
ccs ccs 


Fig. 1—Mean delay characteristics. 
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Fig. 2—Delay distribution. 


up to about z = 2. The three-parameter results* are seen to be in- 
distinguishable from the E-R results up to z = 2.0 and close up to 
about z = 3. In all cases, the E-R results tend to lie between the two- 
and three-parameter results. We see in Fig. 1(b) that the results differ 
greatly for z = 4. The results of Ref. 6 show that fixing the equivalent- 
random mean and variance for a renewal process does not necessarily 
tightly tie down the blocking in the BCC case. We are observing the 
same phenomenon here. In fact, nonnegligible discrepancies between 
the E-R and three-parameter results for larger z’s can occur despite 
a matching of three moments. 

Specific delay distributions are shown in Figs. 2(a) and 2(b). An 
observed property of the results are that the w parameter (5) for the 
two-parameter case exceeded the w parameter of the E-R model which, 
in turn, exceeded the w parameter for the three-parameter case. This 
explains the slope differences. This also tends to order the 7 = 0 
results as shown. In many cases, the A parameter of the distribution 


*In order to avoid severe numerical problems in computing the three-switch 
parameters, double precision was used in eqs. (20) through (24). 
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(8) for the two-moment match exceeded the A parameter for the E-R 
model which, in turn, exceeded the A parameter for the three-param- 
eter match. From these figures, we again observe the closeness of 
results for 2 = 2 and the large differences for z = 4. 

Figures 3(a) and 3(b) plot the load service relationship P(delay > 2.5 
seconds) versus offered load for each of the three models. The compari- 
sons again exhibit the same characteristics that were seen for the mean 
delay results [ Figs. 1(a) and 1(b) ]. 

These results show the extreme sensitivity of the queuing system 
performance to the peakedness of the input process. They also give some 
insight into the region where two- and three-parameter results are 
expected to be closest, i.e., low peakedness and high congestion. When 
seeing the discrepancies between the two- and three-parameter matches 
and the E-R model, we may question which is the bench mark. If the 
peakedness arises from alternate routing, the E-R model seems basic 
since it is an overflow model which has been shown to accurately 
describe superposition of overflows.! This has led to its wide use in 
analyzing blocking in trunking networks. It should also be noted that 
the parameters of the interrupted Poisson process have been chosen to 
match the moments of the E-R model. 


E-R MODEL 
—-W-— 2-PARAMETER 
——-—— 3—PARAMETER 











P (DELAY > 2.5) IN PERCENT 








"200 300 400 500 600 700 100 200 300 400 500 600 
ccs ccs 


Fig. 3—Load service characteristics. 
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E—-R MODEL 
CALL CONGESTION 
——— TIME CONGESTION 





P (DELAY > 2.5) IN PERCENT 














Fig. 4—Call and time congestion. 


Although there are discrepancies between the two- and _ three- 
moment matches and the E-R model for high z’s, the IPP is a close 
approximation to the E-R model for a wide range of practical 2’s. 
Furthermore, it should be emphasized that it provides a convenient 
method of analysis using birth and death equations (and simplified 
Laplace-Stieltjes transform) in many cases where the E-R model is 
intractable. 

Figures 4 and 5 result from applying (33) to the example under 
consideration using the E-R model. Figure 4 shows the load service 
relationship for both call and time congestion. The call congestion 
results, taken from Figs. 3(a) and 3(b), are reproduced here for com- 
parison. It is seen that the time congestion (TC) results consistently 
fall below the call congestion (CC) results.* Figure 5 shows the CC- 
to-TC ratio (w/p) as a function of peakedness. While for a given load 
the time and call congestion can differ substantially, the decrease in 
load that makes up the difference may be relatively small. 


*It_ has been shown by R. P. Marzec that for the Interrupted Poisson Pro- 
cess, Pr[TC > 0] < Pre CC > 0]. This implies p/w S 1, which in turn implies 
Pr[TC > T] S$ Pr[CC > T]. 
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2.0 





P(CC>T)/P(TC> T) 








LOAD (CCS) 


Fig. 5—Call-to-time-congestion ratio. 
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APPENDIX 
Time Congestion 

Let &(t) denote the state of the system (number of customers waiting 
or being served at time ¢) and let P,(t) = Prlé() = k]. Moreover, 
define £, = &(t;) (ie., the number of customers in the system just 
prior to the arrival of the nth customer). Takdcs shows (Ref. 3, 
Theorem 1, p. 148) if » < 1 then the limiting distribution 


lim P[t, =k]=P, (k=0,1,-*°) (36) 


n->cO 


exists and is independent of the initial distribution. Furthermore, he 
shows that 


P, = Aw" (k=N,N+1,-°>), (37) 


where A is given by (8) and w by (5). It is also true that the above 
holds for k = N — 1, i.e., 


P, = Aw" (k=N—-1,N,N+1,--:). (38) 


If we denote by 7, the waiting time of the nth customer, then the 
equilibrium delay distribution, 


W(x) = lim W,(x) = lim P[n, S x], 
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Mot Ss pf? vege Naa) e® 
= . . Nugy Peds 
Wa) = Y Pit  P, [pce A Nuwdy, (89) 


which reduces to (4). We have 
N-1 o 
Wz) = 2 P3+ Dd P;I;(2), 
j=0 j=N 
where J;(x) represents the integral in (39). The complementary dis- 
tribution, W(z), is given by 
W(t) =1- W@) = ¥ Plt - L@)]. (40) 
jc 
If p < 1 and the interarrival distribution F(x) is nonlattice, then the 
equilibrium time congestion probabilities given by 


Pi=limP,(t) (k =0,1,--:) (41) 
£->00 


exist and are independent of the initial state. Furthermore, 
Pi = pPx-1 (kK=N,N+1,-:-). (42) * 


At this point, we are able to evaluate the equilibrium time congestion 
distribution under the hypothesis p < 1, and F(x) is not a lattice 
distribution. 

Denote 7(t) as the waiting time of a fictitious arrival at time ¢ and let 


W(t, x) = Pr[n@® S x]. (48) 
We have 


Wz) = & Pi) + SPD [even TAY Nandy (44) 


since y(t) = Oif é(t) < N. And if é(t) = 7 2 N, then fictitious arrival 
must wait for 7 + 1— N successive departures. These departures 
follow a Poisson process with intensity Nye. Taking the limit and using 
(41) we have 
N—-1 ee) 
W*(z) = lim W(t, 2) = © P37 + DY Pil,(z). (45) 
t->00 j=0 j=N 
Using (42), the complementary distribution is given by 
We) =o Y Pell — L()]. (46) 
= 


*See Theorem 2, p. 153 of Ref. 3. 
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From (88), (40), and (46): 


W*(e) = = Wa). (47)* 


This result is given by Ref. 7, p. 229, for a single-server queue; the 
multiserver case is given here for completeness. 
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Low-Loss Splices in Optical Fibers 


By R. M. DEROSIER and J. STONE 
(Manuscript received March 20, 1973) 


Several methods are reported for making splices in optical fibers. The 
methods have application to both liquid-core and solid-core fibers and have 
been demonstrated for liquid-core fibers. The lowest-loss splice consists of 
an inserted glass pin and an outer sleeve. Best repeatable results are 0.4 dB 
loss in the splice. A splicing device has been constructed which provides 
automatic alignment of the components and automatic assembly for several 
fibers at once. The technique may be directly extended to multiple splicing 
as for fiber cables. 


I, INTRODUCTION 


Several methods have been demonstrated for making splices in 
optical fibers. These have included fused butt joints,!? sandwiching of 
the fiber ends between grooved lucite blocks,*+ and mounting in a 
snug-fitting sleeve.> These techniques have been applied to multimode 
and single-mode fibers. The most desirable splice in an optical fiber 
would be one which can be made quickly and simply, causes negligible 
loss at the junction, and does not have a bulky connector at the 
junction. Furthermore, it should be made with a technique which is 
applicable to simultaneous splicing of a number of fibers in a bundle. 

As a step toward this goal, we have constructed a splicing device 
which has been used to make several types of splices in multimode 
optical fibers. It can be used to make splices in both solid-core and 
liquid-core optical fibers, although we have demonstrated its use here 
only for liquid-core fibers. Its extension to simultaneously splicing a 
number of fibers is obvious and a feature to permit this has been 
included, although not demonstrated. Our best results, which have 
been obtained repeatedly, are 0.4 dB loss in the splice. 


II. DESCRIPTION OF SPLICING DEVICE 


The splicing device was built to be used in several different ways: 
(t) To make temporary butt joints for a number of fibers. 
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(77) To make temporary splices for a number of liquid-core fibers 
by simultaneously inserting transparent index-of-refraction- 
matched glass pins in each splice.® 

(iti) To make permanent splices for a number of fibers by sealing a 
snug-fitting glass sleeve on each splice. 

(tv) To make permanent splices for a number of liquid-core fibers 
by inserting glass pins and sealing on snug-fitting glass sleeves. 


In all of the above applications it was intended that the device be 
capable of carrying out its function by aligning the ends to be spliced 
without any detailed alignment or separate adjustment for each fiber. 

A photograph of the splicing device is shown in Fig. 1. It consists of 
two sliding carriers, shown in Fig. 2, which travel on snug-fitting slide 
bearings on precisely aligned drill rod tracks. Each of the carriers has 
several grooves in it into which fibers may be dropped. The cover clamp 
is slid in on top to hold the fibers snugly in place without crushing them. 

A pan which can be slid up and down is filled with index-matching 
liquid. In the case of liquid-core fibers this is the core liquid. When the 
pan is raised up the fibers are immersed in liquid, and when it is 
dropped the fibers are above the liquid surface. In the center of the 





Fig. 1—Fiber splicing device. A, sliding carrier; B, immersion pan; C, pin holder; 
D, removable clamp for pins; E, removable clamp for fibers. 
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Be 2—Detail of fiber splicer sliding carrier and pin holder. Components are not 
to scale. 


pan is a pedestal with grooves in it. The glass pins to be inserted into 
the fibers sit in these grooves and are aligned by them in such a manner 
that, with the pan raised to its highest position, when the two sliding 
carriers are brought together the pins are coaxial with the fibers to be 
spliced. A cover clamp is then dropped in to hold the pins in place. If 
pins are not used, this portion of the device is ignored. As can be seen 
from the construction of the device, when a splice has been completed, 
the spliced fiber can be lifted out of the splicing device by simply lifting 
off the clamps and then raising the fiber. If a number of fibers have 
been spliced simultaneously, each of the fibers is now an independently 
spliced fiber, with only a slender sleeve adding to its bulk. 


III. FABRICATION OF A SPLICE 


The discussion here will be given in terms of a single splice including 
a pin and sleeve. The two fiber ends to be joined are positioned in the 
aligning grooves of the sliding carriers and clamped in place. The 
grooves are 200 microns wide at the bottom and can be used with 
fibers of outside diameter of about 175 to 200 microns. An excess 
length of fiber is left extending from the clamp. The splice end is then 
obtained by scoring the fiber at the point where it is to be broken and 
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Fig. 3—Intermediate stage of splicing showing glass pin inserted into fiber core. 
The exposed end of the fiber is more irregular than what is used in an actual splice. 


then pulling to break it. A satisfactory end is obtained in this manner, 
as can be seen in Fig. 3. The fiber is broken when immersed in liquid 
and kept immersed until the fiber has been pinned in order to avoid 
formation of a bubble at the junction. 

The fiber pin is made from glass rod. Its index of refraction must be 
higher than that of the fiber cladding to provide guidance. A small 
index mismatch between liquid and pin will give a negligible reflection 
loss (a mismatch of 1 percent gives a reflection at normal incidence of 
10-*). However, it is better if the pin index of refraction is slightly 





4 80um 


Fig. 4—Assembled splice including: A, glass pin; B, fiber cladding; C, outer sleeve. 
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Fig. 5—Finished splice with epoxy-sealed sleeve. 


higher rather than slightly lower than that of the liquid, since for the 
latter case there will be some mode conversion transferring energy to 
the cladding. The fibers used had a quartz cladding index of refraction, 
np = 1.468, and a tetrachloroethylene core, np = 1.505, and the glass 
plug was made from an ordinary commercial glass, np = 1.52. The 
pins were made by stretching the rod after softening in a flame. The 
core diameter of the fiber was about 100 microns, and loose-fitting 
cylindrical plugs of about 80 microns diameter were obtained in lengths 
of about 6 mm. It was found that loose-fitting plugs did not give extra 
loss and were easier to insert than tight-fitting plugs. (Any attempt 
to make a permanent splice by jamming a plug into the fiber ends was 
abandoned because the fiber broke from the pressure of the plug.) 
Furthermore, keeping the plugs undersized avoids the necessity for 
making them tapered. The plugs were cleaned with ‘‘Windex D’’* before 
using. The pin was placed on the center pedestal of the splicing device 
and clamped in place. The two sliding carriers were then brought 
together and the pin entered both fiber ends. 

Due to a small amount of residual play and misalignment ( 25 
microns) it was necessary to manipulate the device slightly in order to 
insert the pin. It is this limitation which prevents the making of 
simultaneous splices. Further shop work should eliminate the residual 
errors. Figure 3 shows a pin which has been inserted into a fiber. 

The outside sleeve is made of glass. Its composition and optical 
quality are unimportant, and other materials could be used. The 
sleeve is made by stretching glass tubing after softening in a flame. 

A length of this tubing is slipped over one of the fiber ends and slid 
out of the way but kept between the sliding carriers before insertion 
of the fiber end in the splicing device. When the pin connection has 


* Copyright 1970, The Drackett Products Co., Cincinnati, Ohio. 
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Fig. 6—Test setup for measuring splice loss. 


been completed, the sleeve is slid over the splice, and then the outer 
clamp and the clamp on the sliding carrier blocking the sleeve are 
removed. The third clamp is then removed and the completed but 
unsealed splice is lifted out. The splice at this stage is shown in Fig. 4. 
The final step is to seal the sleeve onto the fiber. This is done by placing 
a drop of quick-curing epoxy at each end of the sleeve. The epoxy 
consists of equal parts of Shell Epon 828* and Minit-Cure! hardener. 
It cures in several minutes. The completed splice is shown in Fig. 5. 


IV. MEASUREMENT OF SPLICE LOSS 


The test setup used to measure the loss due to the splice is shown in 
Fig. 6. Light from a He-Ne laser at 6328 A is injected into the fiber. 
Since, as is to be expected, the loss in the splice is mode-dependent, a 
realistic measurement is obtained only for a mode distribution charac- 
teristic of that existing in the fiber in its “steady state,” ie., a long 
distance from the beginning of the fiber, such that the mode distribu- 
tion is independent of light-launching conditions. Therefore, the splices 
were made after about 100 m of fiber since previous measurements have 
shown that the steady-state condition has been obtained. Furthermore, 
it is necessary to strip cladding energy both before and after the fiber. 

Splice loss was then obtained by measuring the fiber output without 
a splice and comparing it with the output with an intervening splice. 


V. RESULTS 


In Table I we summarize results obtained for a simple butt joint 
and two types of permanent splices. It can be seen that the best results 


* Manufactured by Shell Petroleum Co., New York. 
t Manufactured by Allaco Products, Inc., Braintree, Massachusetts. 
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TaBLE I—Spuiicina Loss OBTAINED FOR VARIOUS TYPES OF SPLICES 


Two fiber ends Two fiber ends Two fiber ends 
with no plug aligned by an aligned with a plug 
Method (butt joint) outer glass sleeve and outer sleeve 
Average results 1.8 dB 1.2 dB 0.53 dB (19 splices) 
Best results 0.86 dB 0.70 dB 0.40 dB (6 splices) 


are obtained using a plug and sleeve combination. The best results of 
0.40 dB are repeatable, and compare favorably with other reported 
techniques. They are slightly better than any reported for multimode 
fibers, and are exceeded only by Someda’s best results for single-mode 
fibers.4 While Someda’s technique presumably should give as good 
or better results for multimode fibers, this has not actually been 
demonstrated. 

The epoxied splices have been found to be quite strong and, under 
tension, are stronger than the fiber itself which will break first when 
stretched. The splices have been observed over several weeks with no 
observable increase in loss. 


VI. SUMMARY 


A simple mechanical device has been constructed and used to make 
splices in liquid-core optical fibers. This device is also suitable for 
splicing solid-core fibers. The device makes a prealigned splice con- 
sisting of a transparent glass plug and an outside sealed-on sleeve. 
Repeatable results of 0.4 dB loss have been obtained. These compare 
favorably with other reported results. The device may also be used to 
make splices either with the outer sleeve or pin alone. However, in 
either case the losses are higher. The method is directly extendable to 
making multiple splices as, for example, in a cable. The resultant 
splices are compact and independent of each other. 
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A Note on Optimal Approximating 
Manifolds of a Function Class 


By ARUN NETRAVALI 
(Manuscript received March 30, 1973) 


Concept of n-width and extremal subspaces, first introduced by Kolmo- 
gorov, plays an important part in mathematical problems of approximation 
of classes of functions and in engineering problems of signal representation 
and reconstruction. In this short paper, explicit expressions for n-width 
and extremal subspaces are obtained for a class which is of some engineer- 
ing wmportance. 


I. INTRODUCTION 


Kolmogorov! first introduced the concept of n-width as a measure 
of the degree of approximation of a set 2 of a normed linear space X by 
linear manifolds of finite dimension. If S% is an n-dimensional subepace 
of X, then its deviation H(Q, 9%) from Q is defined as 


E(Q, 910) = sup { inf lle — yll} (1) 
zEQ ye 
and the n-width 
d,(Q) = inf {H(Q, 9%): Mm C X, dim M = n}. (2) 


If the lower bound in (2) is attained by ”,, then IM, is called the 
n-dimensional extremal subspace. 

Since Kolmogorov, several authors, including Lorentz,? Tihomirov,? 
Mitjagin,* Golomb,’ and Jerome,® have obtained results on n-widths 
and extremal subspaces of several important function classes. Golomb 
has obtained expressions for n-widths and the narrowest subspaces for 
ellipsoids determined by a linear, nonnegative, self-adjoint operator in 
a Hilbert space. In this paper, these results are extended to the case of 
a class formed by the intersection of an ellipsoid and a unit sphere. 
These classes are important in signal representation and reconstruction 
problems whenever the magnitude of signals has to be constrained 
because of certain physical reasons. 
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II. THE MAIN RESULT 


Following Golomb, if 91% and M2 are subspaces of a complex Hilbert 
space, H, with the inner product (-, -), let us call 911 narrower than 
We (written: Wi <«K Me) whenever Wt 1) Me ¥ ¢, > being the null 
space and 9% the orthogonal complement of 911 in H. The class Q 
under consideration is defined by 


% = {f: f € D(A), (Af, f) S K}, (3) 
m= {fF f=, (4) 

and 
Q = Qy ft) Qe, (5) 


where A is a linear, nonnegative [i.e., (Af, f) = 0 for all f € D(A) ], 
not necessarily bounded, self-adjoint operator with domain D(A) 
which is dense in H. Let \ — E) be the spectral family of A which is 
continuous from the left and &) be the range of F). 

The case 2; C Q is not possible, and the case 22 C , leads to the 
following trivial result: 

E(Q, m) = 1, if M+H 
= 0, if m= dH. (6) 

Therefore, these two cases are not considered below. It is also assumed 
that? omin < 67%, omin < K < 6~?, where comin is the minimum value of 
the spectrum of A. 


Theorem 1: 


(1) EQ, = [ K = omin ; 


ite =. in| ; if WW K 65-2, (7) 


ee 
(ii) E(9, 1) = {ere PS (8) 
Proof: By definition, 
MK 6-3 MLN &- ¥ ¢. 
Then let 97+ 1.) &;-: = G and consider the following two cases: 


é-? — Omin 


(A) Genin iS not empty. 
In this case, take fo € 6&3 such that 


fo= fit fr, fri EG, fa © Sominy (9) 
fll? = 5", (10) 


Tif K <omin, Q will be empty. 
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and 


P eonind if Pe,,,.f1 ¥ 0 


Le | Peominf1|| (11) 


= f-e otherwise, 


where e is any unit vector in 6,,,,, and 
£= v1 — ||Pet. Sill? — |] PsominS ll. (12) 


This construction of fo is illustrated geometrically in Fig. 1, where 
OA is a vector in G and OB is along the subspace spanned by the pro- 
jection P¢,.iJ1. Then fo is the vector sum of f; and a vector f2 along 
OB such that fo has unit magnitude. 

It is easy to see that 


l| foll? 


I 


lf: + fell? 
Peewee =F fri? a | Pexiinf ll? 
1 (13) 


ll 


Il 


_ 7 UNIT CIRCLE 





Fig. 1—Geometrical construction of fo. 
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and 


§-2 
(Afy fo) =f raL| Es fll?) 


Omi 


GminL||Peominft + fell?) + 8-?[||PediinSsll?] 
ominLL — ||Pesninfall?] + 6-?L|| PesminS sll? 
| P eas al|?L8-? ae o min] + Omin 
S K— Omin + Omin = K. (14) 


IA IIA IA 


Hence fo € 2. Now 
|| Pefoll? 
=(fi + Pefe, fi + Pofe) 
= (fi, fi) + (Pafe, Pafe) + 2(fi, Pefe) 


= (fi, fi) + (Pefe, Pafe) 


2¢ 
+ |Peonafill (fi, PoP tomind'1)s if Péonnfi #0 (15) 


> Al? = Soe. (16) 


— Omin 

Also, if Peoninft = 0, then the third term in the RHS of eq. (15) is 

equal to zero and (16) still holds. Equations (13), (14), and (16) 
together imply that 

EXO, m) = Bf mw) = A, 


(B) SeminiS emnpty. 


(17) 


— Omin 


Since omin 1s a point in the spectrum, a sequence exists of nonincreas- 
ing and nonnegative real numbers {6;} such that each 6; is in the spec- 
trum of A and the following conditions are met: 


lim 6; = omin, (18a) 


5; < K, for all z, (18b) 


and &;,is not empty (i.e., it contains more than the zero element) for 
any 6;. Then, for each 6; using the same construction as in (A) above 
(using &3, instead of &,,;,), it can be shown that 
K — 6; 
EX(Q, NM) 2 p76, (19) 
However, since (K — 6,)/(6-? — 6;) isa nondecreasing sequence of posi- 
tive numbers tending to (K — omin)/(67~? — omin), 


EQ, mM) = oan (20) 


— Omin 


This proves (2). 
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(iz) Consider now the case in which I = 6;-2. For any f€ Q, 
E*(f, 65-2) = ||f — Ea-fll? 
= f° ates] 


1 ) 
= aS po (A a min) A(|| Ef ||? ) 
1 ) 
Sa [OR om (HES 21) 
K—- Omin 
= 92 — Smin oe 
Equation (22) follows from (21) because, for all f € Q, 
[Oral sie s K (23) 
and 
fO atiznsie] = Wf? = (24) 
It follows from (22) that 
K — Omin 3 
Joe 
E(Q, &s-2) S 1s = nin | ; (25) 
Theorem 2: 
. rae iF 
B(O, 6.~) = {Fee} (26) 
— Omin 
af and only tf 6-* 1s in the spectrum of A and, tn this case, 
B(O, tt) > {aie} (27) 


af MK E572. 
Proof of this theorem is similar to the proof of Theorem 2 in Golomb® 
and is therefore omitted. 
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A Finline Radiator 


By D. C. HOGG and W. E. LEGG 
(Manuscript received May 11, 1973) 


I. INTRODUCTION 


Tapered finline might constitute a radiating element compatible 
with planar technology and therefore be suitable for antenna arrays 
fed by integrated circuitry. We have constructed such an element, not 
by photoetching a substrate but by mounting a tapered fin in a circular 
waveguide such that it is fed in Robertson’s traditional way, and then 
applying a dielectric ‘“‘substrate”’ to the fin. Measurements of radiation, 
patterns at 18.5 GHz are given for the prototype element using various 
“substrate” thicknesses. It is found that the element has considerable 
gain, and that performance improves with the thickness of the simu- 
lated substrate. 
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Fig. 1—Finline radiator. 
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Fig. 2—Radiation patterns at 18.5 GHz with no dielectric applied. (a) H-plane 
pattern. (b) E-plane pattern. 
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Fig. 3—Radiation patterns at 18.5 GHz with dielectric thickness of 10 mils. 
(a) H-plane pattern. (b) E-plane pattern. 
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II. DESIGN 


The impedance and power flow along a fin in circular waveguide has 
been calculated by S. P. Morgan? but, to the author’s knowledge, no 
information is available regarding radiation properties. Therefore, the 
simple device shown in Fig. 1 was built for initial tests. The dominant- 
mode circular waveguide is matched into the finline gap via a taper. 
After exit from the waveguide the finline is gently expanded to a 23 
inch ‘aperture’ which amounts to about four wavelengths at 18.5 
GHz. The fin is 20 mils thick and the finline per se has a gap of 6 mils, 
resulting in an impedance of about 70 ohms. The taper is such that the 
phase error over the aperture, in the plane of the fin, is about 7/2; the 
flare angle is 30 degrees. The return loss exceeds 20 dB over, at least, 
a ten-percent band. 


III. RADIATION PATTERNS 


Figure 2 shows the H- and E-plane radiation patterns (orthogonal 
to and in the plane of the fin, respectively) ; in this casé no dielectric 
(substrate) is applied. 

The E-plane pattern (in the plane of the fin) has several shoulders; 
but these are not unexpected? for phase errors of the order 7/2. Figures 
3, 4, and 5 show patterns where polyethylene substrates of thickness 
10, 15, and 27 mils, respectively, were applied over an entire side of the 
fin structure, i.e., also within the waveguide. Note that the E-plane 
patterns remain essentially constant with increasing substrate thickness. 

The H-plane patterns (in the plane orthogonal to the fin) are inter- 
esting. In the first place, the pattern of Fig. 2a is more well-behaved 
and somewhat narrower than the E-plane pattern, Fig. 2b. This means 
that the field spreads considerably in the transverse plane resulting in 
a sizeable effective aperture dimension. Moreover, as shown in Figs. 
3a, 4a, and 5a, the addition of substrate material narrows the beam- 
width of the H-plane pattern. Table I lists the 3- and 10-dB beamwidths 
of the H-plane pattern as a function of substrate thickness. 

The gain of the 27-mil version is about 15 dB. 


IV. CONCLUSION 


The tapered finline radiator appears to be a reasonable candidate 
for an array element where planar technology is involved. Element 
beamwidths appropriate for certain tasks (such as illumination of the 
contiguous United States from synchronous orbit where beamwidths of 
the order of 10 degrees are desirable) appear feasible by suitable choice 


1252 THE BELL SYSTEM TECHNICAL JOURNAL, SEPTEMBER 1973 


ANGLE IN DEGREES ANGLE IN DEGREES 
350 O10 








ou AKA PINS 
180 130 


Fig. 4—Radiation patterns at 18.5 GHz with dielectric thickness of 15 mils. 
(a) H-plane pattern. (b) E-plane pattern. 
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Fig. 5—Radiation patterns at 18.5 GHz with dielectric thickness of 27 mils. 
(a) H-plane pattern. (b) E-plane pattern. 
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TABLE IJ—BEAMWIDTHS OF H-PLANE PATTERN 


H-Plane Beamwidths 


Substrate Degrees 
(Polyethylene) 
Thickness-Mils 3 dB 10 dB 
0 30 48 
10 23 36 
15 20 34 
27 20 32 


of flare angle and aperture. Of course, an objective of the design is that 
the radiator be integrable with microstrip circuitry; this requires a 
microstrip-to-finline (slotline) coupler, perhaps of the type described 
by S. B. Cohn. 
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